Today, we are trying Deep Neural Networks on many [previously] unsolved problems. Image and language recognition with CNNs and LSTMs has become a standard. Machines can classify images/speech/text faster, better and much longer than humans.
There is breakthrough in computer vision in real-time, capable to identify objects and object segments. That’s very impressive, it enables self-driving cars, and in-doors positioning without radio beacons or other infrastructure. The machine sees more than human, because the machine sees it all in 360 degrees. And the machine sees more details simultaneously; while human overlooks majority of them.
We created some new kind of intelligence, that is similar to human, but is very different from human. Let’s call this AI as Another Intelligence. The program is able to recognize and identify more than one billion human faces. This is not equivalent what humans are capable to do. How many people could you recognize/remember? Few thousands? Maybe several thousands? Less than ten thousands for sure (it’s the size of small town); so 1,000,000,000 vs. 10,000 is impressive, and definitely is another type of intelligence.
DNNs are loved and applied almost to any problem, even previously solved via different tools. In many cases DNNs outperform the previous tools. DNNs started to be a hammer, and the problems started to be the nails. In my opinion, there is overconfidence in the new tool, and it’s pretty deep. Maybe it slows us down on the way of reverse engineering the common sense, consciousness…
DNNs were inspired by neuroscience, and we were confident that we were digitally recreating the brain. Here is cold shower – a man with a tiny brain – 10% size of the normal human brain. The man was considered normal by his relative and friends. He lived normal life. The issue was discovered accidentally, and it shocked medical professionals and scientists. There are hypothesis how to explain what we don’t understand.
There are other brain-related observations, that threaten the modern theory of brain understanding. Birds – some birds are pretty intelligent. Parrots, with tiny brains, could challenge dolphins, with human-sized brains, and some chimps. Bird’s brain is structured differently from the mammalian brain. Does size matter? Elephants have huge brain, with 3x more neurons than humans. Though the vast majority of those neurons are within different block of the brain, in comparison to humans.
All right, the structure of the brain matters more than the size of the brain. So are we using/modeling correct brain structure with DNNs?
Numenta is working on reverse engineering the neocortex for a decade. Numenta’s machine intelligence technology is built on the own computational theory of the neocortex. It deals with hierarchical temporal memory (HTM), sparse distributed memory (SDM), sparse distributed representations (SDR), self-organizing maps (SOM). The network topologies are different from the mainstream deep perceptrons.
It’s fresh stuff from the scientific paper published in free frontiers magazine, check it out for the missing link between structure and function. “… remarkably intricate and previously unseen topology of synaptic connectivity. The synaptic network contains an abundance of cliques of neurons bound into cavities that guide the emergence of correlated activity. In response to stimuli, correlated activity binds synaptically connected neurons into functional cliques and cavities that evolve in a stereotypical sequence toward peak complexity. We propose that the brain processes stimuli by forming increasingly complex functional cliques and cavities.”
When human is shown a new symbol, from previously unseen alphabet, it is usually enough to recognize the other such symbols, when shown again later. Even in the mix with other known and unknown symbols. When human is shown a new object, for the first time, like segway or hoverboard, it is enough to recognize all other future segways and hoverboards. It is called one-shot learning. You are given only one shot at something new, you understand that it is new for you, you remember it, you recognize it during all future shots. The training set consists of only one sample. One sample. One.
Check out this scientific paper on human concept learning with segway and zarc symbol. DNNs require millions and billions of training samples, while the learning is possible from the only one sample. Do we model our brain differently? Or are we building different intelligence, on the way of reverse-engineering our brain?
These are two models of the same kart, created differently. On the left is human-designed model. On the right is machine-designed model within given restrictions and desired parameters (gathered via telemetry from the real kart from the track). It is paradigm shift, from constructed to grown. Many things in nature do grow, they have lifecycle. It’s true for the artificial things too. Grown model tends to be more efficient (lighter, stiffer, even visually more elegant), than constructed ones.
How to generate? Good start would be to use evolutionary programming, with known primitives for cells and layers. Though it is not easy to get it right. By evolving an imaginable creature, that moves to the left, right, ahead, back, it is easy to get asymmetrical blocks, handling the left and right. Even by running long evolutions, it could be hardly possible to achieve the desired symmetry, observed in the real world, and considered as common sense. E.g. the creature has very similar or identical ears, hands, feet. What to do to fix the evolution? To bring in the domain knowledge. When we know that left and right must be symmetrical, we could enforce this during the evolution.
The takeaway from this section – we are already using three approaches to AI programming simultaneously: domain rules, evolution and deep learning via backpropagation. Altogether. No one of them is not enough for the best possible end result. Actually, we even don’t know what the best result is possible. We are just building a piece of technology, for specific purposes.
The above approach of using domain rules, evolution and deep learning via backpropagation altogether might not be capable to solve the one-shot learning problem. How that kind of problems could be solved? Maybe via Bayesian learning. Here is another paper on Bayesian Framework, that allows to learn something new from few samples. Together with Bayes we have four AI approaches. There is a work on AI, identifying five [tribes] of them.
The essense is in how to learn to learn. Without moving the design of AI to the level when AI learns to learn, we are designing throw-away pieces, like we did with Perl programming, like we do with Excel spreadsheets. Yes, we construct and train the networks, and then throw them away. They are not reusable, even if they are potentially reusable (like substituting the final layers for custom classification). Just observe what people are doing, they all are training from the very beginning. It is the level of learning, not the learning to learn – i.e. it’s throw-away level. People are reusable, they could train again; while networks are not reusable.
The Master Algorithm is the work, that appeals to the AI creators, who are open-minded to try to break through the next level of abstraction. To use multiple AI paradigms, in different combinations. It is design of design – you design how you will design the next AI thing, then apply that design to actually build it. Most probably good AI must be built with special combination of those approaches and tools within each of them. Listen to Pedro Domingos for his story, please. Grasp the AI quintessence.
Start from this cool comparison of Mathematics and Physics by Richard Feynman. Physicists are always about the special case. Mathematicians are always about the general case. Physicists do reverse engineer the world; recreating the technologies, available in the Universe. Physicists even think beyond the Universe…
Continue with these ruminations about Mathematics by Stephen Wolfram – was mathematics invented or discovered. He thinks that the math is already there, we just need to get to those spaces.
Here are details on the Computing Theory of Everything, by Stephen Wolfram. Like Galileo Galilei invented the telescope to observe and discover the far space, Wolfram invented and invents tools to discover the math, all those spaces. It is not combinatoric mess, as the spaces could be shaped nicely, depending on the laws within. Look at this amazing Rule 30, look at this annoying Rule 184.
Think of forthcoming Quantum Computing, which is closer to what Feynman foresaw about machinery without mathematics (watch first video, from 6:00 to 7:30). Why we need an infinite computational power, based on mathematics & logic, to figure out what happens in the tiny place in space? Pretty modern supercomputer needs few hours to simulate 10^11 individual atoms, which is ~10^11 times smaller than the number of atoms in only 1 gram of iron (Fe)…
But about simulating the new worlds, at the level of individual atom. We could build a simulation, and it will go with mathematics. We just need to squeeze the computational power from the physical universe.
Are mathematics and physics converging?
PS. Everything above physics in understood. Chemistry deals at bigger sizes. And so on upwards to huge sizes… till the edge of the Universe, where we still don’t understand. But maybe the Math will help here?
…Then the Pterodactyl burst upon the world in all his impressive solemnity and grandeur, and all Nature recognized that the Cainozoic threshold was crossed and a new Period open for business, a new stage begun in the preparation of the globe for man. It may be that the Pterodactyl thought the thirty million years had been intended as a preparation for himself, for there was nothing too foolish for a Pterodactyl to imagine, but he was in error, the preparation was for Man… — Mark Twain
The Man. The man who won Tour de France seven times. Having reached the human limit of physical capabilities, he [and others] extended them. He did blood doping (by taking EPO and other drugs, storing own blood in the fridge, and infusing it before the competition for boosting the number of red blood cells, thus performance). He [and others] took anti asthmatic drugs to increases performance on endurance. And so on, so on. There are Yes or No answers from Lance himself from Oprah’s interview.
Is Lance cheater? Or is Lance hero? I consider him a hero for two reasons. First, he competed against the same or similar. Second, he went beyond the human limits, cutting-edge thinking, cutting-edge behavior, scientific sacrifice, calculated or even bold risk.
What could be said about all other sportsmen? I think the sporting pharmacology is evolutionary logical stage for the humankind to outperform our ancestors, to break the records, to win, and continue winning. If sportsmen are specialized in competing, and society wants them competing, then everything all set. Evolution goes on, biological meets artificial chemical. It improves the function, it solves the problem. Though it slightly distance biological ourselves from what we though we were.
It happens that people lose body parts. It is right way to go to give them missing parts. It’s still very complicated, the technologies involved are still not there, but good progress has been made. There are new materials, new mechanics, new production (digital manufacturing, 3D printing), new bio-signal processing (complex myogram readings), new software designed (with AI), and all together it gives tangible result. Take a look at this robot, integrated with the man:
Some ethical questions emerge. The man with prosthetic body part is still a biological being? What is a threshold between biological parts and synthetic parts to be considered a human being? There are people without arms and legs, because of injuries or because of genetic diseases, like Torso Man. We could and should re-create the missing parts and continue living as before, using our new parts. Bionic parts must evolve until they feel and perform identically to original biological parts.
It relates to invisible organs too. The heart, which happen to be a pump, not a soul keeper. People live with artificial hearts. Look at the man walking out from hospital without human heart. The kidneys, which are served by external hemodialysis machines. New research is performed to embed kidney robots into the body. Ethical questions continue, where is a boundary what we call a ‘human’? Is it head? Or brain only? What makes us human to other humans?
We are defined by our genes. Our biological capabilities are on genes. Then we learn and train to build on top of our given foundation. We are different by genes, hence something that is easy for one could be difficult for another. E.g. since childhood sportsmen usually have better metabolism in comparison to those who grow to ‘office plankton’.
There are diseases caused by harmful mutations on genes. Actually any mutation is bad, because of unpredictable results in first generation with new mutant [gene]. But some mutations are bad from generation to generation, called genetic disease. It is possible to track many diseases down to the genes. There are Genetic Browsers allowing to look into the genome down to the DNA chain. Take a look at the CFTR gene, first snapshot is high-level view, with chromosome number and position; second is zoomed to the base, with ACGT chain visible.
If parents with genetic disease want to prevent their child from that disease, they may want to fix the known gene. Everything else [genetically] will remain naturally biological, but only that one mutant will be fixed to the normal. The kid will not have the disease of ancestors, which is good. A question emerges: is this kid fully biological? How that genetic engineering impacts established social norms?
What if parents are fans of Lance Armstrong and decide to edit more genes, to make their future kid a good sportsman?
Digging down to the DNA level, it is very interesting to figure out what is possible there to improve ourselves, and what is life at all. How to recognize life? How would we recognize life on Mars, if it’s present there?
Here is definition from Wikipedia: “The definition of life is controversial. The current definition is that organisms maintain homeostasis, are composed of cells, undergo metabolism, can grow, adapt to their environment, respond to stimuli, and reproduce.” The very first sentence resonate with questions we are asking…
Craig Venter led the team of scientists to extract the genetic material from the cell (Mycoplasma genitalium), instrumented its genome by inserting the names of 20 scientists and the link to the web site, implanted edited material back into the cell, observed the cell reproducing many times. Their result – Mycoplasma laboratorium – reproduced billions times, passing encoded info through generations. The cell had ~470 genes.
What is absolutely minimum number of genes, and what are those genes, to create life? Is it 150? Or less? And which one exactly? What are their specialization/functions? It’s current on-going experiment… Good luck guys! Looking forward to your research success, and what is Minimum Viable Life (MVL). BTW by doing this experiment, scientists designed new technologies and tools, allowing to model the genes programmatically, and then synthesize them at molecular level.
While somebody are digging into the genome, others are trying to replicate humans (and other creatures) at macro level. Most successful with humanoid machines are Boston Dynamics.
How far we are to make them indistinguishable from humans? Seems that pretty far. The weight, the gravity center, motion, gestures, look & feel are still not there. I bet that humanoids will be first create in military and porn. Military will need robots to operate plenty of outdated military equipment, serve and combat in hazard environments. it’s only old weaponry that require manned control. While new weapons are designed to operate unmanned. Porn will evolve to the level that we will fuck the robots. For military it’s more the economical need. For our leisure it’s romantic need and personal experience.
The size and shape of robots doing mechanical work is so different. From tunnel drilling monsters to blood vessels…
If we look for the commonality in mentioned (and several unmentioned) disrupting technologies, we could select 8 of them (extended and reworked 8 directions of Singularity Univeristy), which stand out:
As we slightly covered Biology, Medicine and Robotics already, more to be said about the rest. But before than, few words about Biotech. We could program new behavior of the biomass, by engineering what the cells must produce, and use those biorobots to clean the landfills around the cities, sewerage, rivers, seas, maybe air. Biorobots also could clean our organisms, inside and outside. Specially engineered micro biorobots could eat the Mars stones and produce the atmosphere there. Not so fast but feasible.
Well, more words about other disrupting technologies. Networks and Sensors next. First of all – it’s about networks between human & human, machine & machine, human & machine. The network effect happens within the network, known as Metcalfe’s Law. Networks are wired and wireless, synchronous and asynchronous, local and geographically distributed, static and dynamic mesh etc. Very promising are Mesh Networks, allowing to avoid Thing-Cloud aka Client-Server architectures, despite all cloud providers pushes for that. Architecturally (and common sense) it’s better to establish the mesh locally, with redundancy and specialization of nodes, and relay the data between the mesh and the cloud via some edge device, which could be dynamically selected.
Sensors will be everywhere. Within interior, on the body, as infrastructure of the streets, in ambient environment, in the food etc. Our life is improved when we sense/measure and proactively prepare. We used to weather forecasts, which are very precise for a day or two. It’s because of huge amount of land sensors, air sensors, satellite imagery. Body sensors are gaining popularity, as wearables for quantified self. There are recommendations for the lifestyle, based of your body readings. It’s early and primitive today, but it will dramatically improve with more data recorded and analyzed. Modern transportation requires more sensors within/along the roads and streets, and cars. It’s evolving. Miniaturization shapes them all. Those sensors must be invisible for the eyes, and fully integrated into the cloths and machines and environment.
3D Printing. The biggest change is related to ownership of intellectual property. 3D model will be the thing, while its replication at any location on demand on any printer will be commodity function. Many things became digital: books, photos, movies, games. Many things are becoming digital: hard goods, food, organs, genome. It’s a matter of time when we have cheap technology capable to synthesize at the atom grid level and molecular. New materials are needed everywhere, especially for human augmentation, for energy storing and for computing.
Nanotech. We learn to engineer at the scale of 10^-9 meter. From non-stick cookware and self restoring paint (for cars), to sunscreen and nanorobots for cleaning our veins, to new computing chips. Nano & Bio are very related, as purification and cleanup processes for industry and environment are being redesigned at nano level. Nano & 3D Printing are related too, as ultimate result will be affordable nanofactory for everyone.
Computing. We’re approaching disruption here, Moore’s Law is still there but it’s slowing down and the end is visible. Some breakthrough required. Hegemony of Intel is being challenged by IBM with POWER8 (and obviously almost ready POWER9) and ARM (v8 chips). Google is experimenting with POWER and ARM. it’s true, Qualcomm is pushing with ARM-based servers. D:Wave is pioneering Quantum Computing (actually it’s superconductivity computing). There is good intro in my Quantum Hello World post. IBM recently opened access to own quantum analog. The bottom line is that we need more computing capacity, it must be elastic, and we want it cheaper.
Artificial Intelligence. AI deserves separate chapter. Here it is.
The purpose of AI was machine making decisions ( as maximization of reward function). But being better at making decisions != making better decisions. Machine decide how to translate English-to-Ukrainian, but not speaking either language. Those programs (and machines) are super screwdrivers, they don’t what to do, we want them to do, we put our want into them.
AI is different intelligence, human cannot recognize 1 billion humans, even really having seen them all many times. AI is Another Intelligence so far. The shape of thinking machines is not human at all: DeepBlue – chess winner – is a toll black box; Watson – Jeopardy winner – 2 units of 5 racks of 10 POWER7 servers between noisy refrigerators in nice alien blue light (watch from 2:20); Facebook Faces – programs and machines recognizing billions of human faces – it’s probably big racks in data center, Google Images – describing context of the image – big part of the data center (detection of cat took 16,000 servers several years ago); Space Probes – totally different from both humans and black toll boxes in the data centers.
BTW if somebody really spots UFO visiting our planet, don’t expect green men, as organics is poor for space travel, because of dangerous +200/-200 Celsius temperature range, ultra violet and radiation, time needed for travel (even through the wormhole)… That UFO is a robot most probably. Or intelligence on non-biological carrier, which means post-biological species (which is worse for us if so).
Our wet brain operates at 100 Watts, while the copy of the simulation of the same number of cells requires 10^12 Watts. Where on Earth will we get 1 trillion watts just for equivalent of one human intelligence? Even not intelligence, but connectivity of the neurons. Isn’t it ridiculous pseudo architecture? We still did not isolate what we call consciousness, and we don’t know it’s structure to properly model it. Brain scanning is in progress, especially for deeper brain. And this Eureka moment, like we got with DNA, is still to come.
We’re remaining at the center, creating and using machines for mental work, like we created and used/use machines for physical work. Humans with new mental tools should perform better than without them. Google is a typical memory machine, and memory prosthesis. Watson as a layer or a doctor is a reality.
Back from the future, at present we have intelligent machines – governments and corporations. We created those artificial bodies many years ago, and just don’t realize they are true intelligent machines. They are integrated into/with society, with law evolved through precedents and legislation, tailored to different locations and cultures. Culture itself is a natural artificial intelligence. Global biological artificial intelligence emerged on politicians, lawyers, organizations like United Nations and hundreds of smaller international ones. They are all candidates for substitution by programs and machines.
Interesting observation is that most intelligent humans neither harmful nor rulers of others. Hence we could assume that really smart AI will not be harmful to humans, when AI will be approximately at our level. But it’s uncertain about accelerated and grown AI later in time. Evolution will shape AI too, continuing from invisible interfaces with machines right now. We could stop clicking, typing, tapping into machines, and talk to them like we do between ourselves. Today we have three streams of AI: < 3yo AI, Artificial Smartness, Intelligence as a Service.
We are what we eat, hence they will have to eat us? Hm… Real AI will not reveal itself. And most probably they will leave, like we left our cradle Africa…
There were some concerns that we had slowed down, by observations and perception of the daily facts. But it’s also visible that several technologies are booming and disrupting our lives almost on weekly basis. Those 8 mentioned earlier technologies in section It All Together. Those technologies are developing exponentially.
The companies are highly specializing within their niches, performing at global scale. Global economy is changing. Few best providers of the narrow function do it world-wide. E.g. Google is serving search globally, with two others far behind (Baidu and Bing, with artificial restriction of Google in China). Illumina chips are used for gene sequencing (90 percent of DNA data produced). Intel chips are primary host processors in the servers. Nvidia are primary coprocessors and so on. Few companies fulfill the 95+ percent of the needs within some niche. Where this has not happen yet, big disruption is expected soon.
This is pure specialization of work at global scale. Shift from normal distribution to power distribution. Some may say that it’s path to global monopolism, with artificially hold high costs. But in fact it is not, as Google search is free. Illumina is promising full human genome sequenced under $1,000. And Intel still ships new chips according to Moore’s Law, 2x productivity per $1 every 1.5 year.
As global specialization reduces global costs, because same functions and products are produced more efficiently on same resources, it is good for our planet, with limited resources. But here another thing happens, we are not preserving resources, we are using them for creating new technologies, which are expensive, unique, disrupting. Provider of such new technology (and product, service) is not a monopolist, because of small scale/capacity at the beginning. Either they scale or others replicate it, and true leader emerges and make it globally. Also new ways for energy are found, from Sun and wind, and new nuclear too. We’re creating more wealth.
Scaling globally is dramatically easier and cheaper for digital products and services, than for physical/hard or hybrid. It is main motivator for digitization of everything. Software is eating the world, because it is simply cheaper to deliver sw vs. hw. Everything will become software, except the hardware to run the software, and power plants to empower the hardware.
Real life is becoming digital very fast. Why we’re taking photos of our meals and rooms, self faces and legs, beautiful and creepy landscapes, compositions? Why we checkin, express status, emotions for others’ expressed statuses, commenting, trolling and even fighting digitally? We also voting, declaring, reporting, learning, curing, buying and consuming, entertaining digitally too. We’re living digitally more than physically sometimes. Notice how people record the event looking at their smartphone small screen instead of looking at the big stage and experience it better. Some motivation drives us to record it to multiple phones, from multiple locations, aspects, angles, distances, and push it into the internet, and share with others. Then see it all from those recordings, own and theirs. Why is it happening? Why we are shifting to digital over natural? Or digital is new natural, as evolution goes on?
Kit Harington was stopped by cop for speeding. The cop made ultimatum – either driver pays fine, or he tells whether Jon Snow is alive in next season. The driver avoided the speeding ticket by telling the virtual/digital story to the cop. For the cop digital virtual was more important than physical biological. Isn’t it natural shift to new better reality?
Many people live is virtual worlds today. Take American and ask about ISIS. Take Syrian and ask about ISIS. Take Ukrainian and ask about Crimea and Donbass. Take Russian and ask about Crimea and Donbass. Same for Israel and Palestina. People will tell opposite everything. People are already living in virtual worlds, created by digital television and internet. Digitization of life is here already, and we are there already.
Specialization is observed at all levels. Molecules specialized into water, gases, salts, acids. Bigger molecules specialized into proteins and DNA. Then we have cells, stem cell and their specialization into connective tissue, soft tissue, bone and so on. Next are organs. Then body parts. Specialization is present at each abstraction level. At the level of people specialization is known as roles and professions. Between businesses and countries it is industries. Between nations it is economics and politics.
It looks like we are part of the bigger machine, which is evolving with acceleration. We are like cells, good and bad, specialized from vision to thinking. Roads, pipes are like transportation systems for other cells and payload. Internet (copper and fiber) is more like a neural system. Connectivity is a true phenomenon. We are now fully disconnected (and useless) without smartphone, or without digital social network in any form. Kevin Kelly once called it the One. The Earth of many people will evolve into earth of augmented people and machines, they all specialize and unite into the One.
And since the One, it all looks like just a beginning. I feel another One, and more cells-ones, organizing something more complex and intelligent from themselves. If our cells could specialize and unite into 10 trillions and walk, think, write, why it can’t be possible with bigger cells like One, at bigger scale like Galaxy?
The Man is not the last smart species on Earth. In other words, there will be a day, when the Last [current] Man on Earth goes extinct. What will happen faster: transhuman or true AI, that could replicate and grow? I bet on transhuman. Better for humanity too. For now.
This is an IoT story combined from what was delivered during Q1’15 in Stockholm, Copenhagen and Bad Homburg (Frankfurt).
When I first heard Peter Thiel about our technological deceleration, it collided in my head with technological acceleration by Ray Kurzveil. It seems that both gentlemen are right and we [humanity] follow multi-dimensional spiral pathway. Kurzveil reveals shorter intervals between spiral cycles. Thiel reveals we are moving in negative direction within single current spiral cycle. Let’s drill down within the current cycle.
Cars are getting more and more powerful (in terms of horse power), but we don’t move faster with cars. Instead we move slower, because of so many traffic lights, speed limits, traffic jams, grid locks. It is definitely not cool to stare at red light and wait. It is not cool either to break because your green light ended. In Copenhagen majority of people use bikes. It means they move at the speed of 20 kph or so… Way more slower than our modern cars would have allowed. Ain’t strange?
Aircrafts are faster than cars, but air travel is slow either. We have strange connections. A trip from Copenhagen to Stockholm takes one full day because you got to fly Copenhagen-Frankfurt, wait and then fly Frankfurt-Stockholm. That’s how airlines work and suck money from your pocket for each air mile. Now add long security lines, weather precautions and weather cancellations of flights. Add union strikes. Dream about decommissioned Concord… 12 years ago already.
Smartphone computing power equals to Apollo mission levels, so what? Smartphone is used to browse people and play games mainly. At some moment we will start using it as a hub, to connect tens of devices, to process tons of data before submitting into the Cloud (because Data will soon not fit into the Cloud). But for now we under-use smartphones. I am sick of charging every day. I am sick for all those wires and adapters. That’s ridiculous.
Cancer, Alzheimer and HIV still not defeated. And there is not optimistic mid term forecast yet.
We [thinking people] admit that we have stuck in the past. We admit our old tools are not capable to bring us into the future. We admit that we need to build new tools to break into the future. Internet of Things is such a macro trend – building those new tools what would breakthrough us into the future.
We are within 3rd wave of IoT called Identification and at the beginning of 4th wave of IoT called Miniaturization. Those two slightly overlap.
Miniaturization is evidence of Moore’s Law still working. Pretty small devices are capable of running same calculations as not so old desktops. Connecting industrial machinery via man-in-the-middle small device is on the rise. It is known as Machine-to-Machine (M2M). Two common scenarios here: wire protocol – to break into dumb machine’s wires and hook it there for readings and control; optical protocol – read from analog or digital screens and do optical recognition of the information.
More words about optical protocol in M2M. Imagine you are running biotech lab. You have good old centrifuges, doing their layering job perfectly. But they are not connected, so you need to read from display and push the buttons manually. You don’t want to break into the good working machines and decide to read from their screens or panels optically, hence doing optical identification for useful information. The centrifuges become connected. M2M without wires.
Identification is also on the rise in manufacturing. Just put a small device to identify something like vibration, smoke, volume, motion, proximity, temperature etc. Just attach a small device with right sensors to dumb machine and identify/measure what you are interested in. Identification is on the rise in life style. It is Wearables we put onto ourselves for measure various aspects of our activity. Have you ever wondered how many methods exist to measure temperature? Probably more than 10. Your (or my?) favorite wearables usually have thermistors (BodyMedia) and IR sensors (Scanadu).
Optical identification as powerful field of entire Internet of Things Identification requires special section. Continue reading.
Why optical is so important? Guess: at what bandwidth our eyes transmit data into the brain?
It is definitely more than 1 megabit per second and may be (may be not) slightly less than 10 megabit per second. For you geeks, it is not so old Ethernet speed. With 100 video sensors we end up with 1+ Terabyte during business hours (~10 hours). That’s hell a lot of data. It’s better to extract useful information out of those data streams and continue with information rather than with data. Volume reduction could be 1000x and much more if we deal with relevant information only. Real-time identification is vital for self-driving cars.
Even for already accumulated media archives this all is very relevant. How to index video library? How to index image library? It is the work for machines, to crawl and parse each frame and sequences of frames to classify what it there, remember timestamp and clip location, make a thumbnail and give this information to those who are users of the media archives (other apps, services and people). Usually images are identified/parsed by Convolution Neural Networks (CNN) or Autoencoder + Perceptron. For various business purposes, the good way to start doing visual object identification right away is Berkeley Caffe framework.
Ever heard about DeepMind? They are not on Kaggle today. They were there much earlier. One of them, Volodymyr Mnih, won the prize in early 2013. DeepMind invented some breakthrough technology and was bought by Google for $400 million (Facebook was another potential buyer of DeepMind). So what is interesting with them? Well yeah, the acquisition was conditional that Google would not abuse the technology. There is special Ethical Board set up at Google to validate use of DeepMind technology. We could try to figure out what their secret sauce is. All I know is that they went beyond dumb predefined machine learning by applying more neuroscience stuff which unlocked learning from own experience, with nothing predefined a priori.
Volodymyr Mnih has been featured in recent (at the moment of this post) issue of Nature magazine, with affiliation to DeepMind in the references. Read what they did – they build neural network that learns game strategy, ran it on old Atari games and outperformed human players on 43 games!! It is CNN, with time dimension (four chronological frames given to input). Besides time dimension, another big difference to classic CNN is delayed rewards learning mechanism, i.e. it’s true strategy from your previous moves. The algorithm is called Deep Q-learning, and the entire network is called Deep Q-learning Network (DQN). It is a question of time when DQN will be able to handle more complicated graphical screens than old Atari. They have tried Doom already. May be StarCraft is next. And soon it will be business processes and workflows…
Those who subscribed to Nature, log in and read main article, especially Methods part. Others could check out reader-friendly New Yorker post. Pay attention to Nature link there, you might be lucky to access Methods section on Nature site without subscription. Check out DQN code in Lua and DQN + Caffe and DQN/Caffe ping pong demo.
All right with importance of optical identification, hope it’s time to switch back to high-level and continue on the Internet of Things as macro trend, at global scale. Many of you got used to the statement that Software is eating the World. That’s correct for two aspects: hardware flexibility is being shifted to software flexibility; fabricators are making hard goods from digital models.
Shifting flexibility from hardware to software is huge cost reduction of maintenance and reconfiguration. The evidence of hardware eaten by software are all those SDX, Software Defined Everything. E.g. SDN aka Software Defined Networks, SDR aka Software Defined Radio, SRS aka Storage, and so of for Data Center etc. Tesla car is a pretty software defined car.
But this World has not been even eaten by hardware yet! Miniaturization of electric digital devices allows the Hardware to eat the World today and tomorrow. Penetration and reach of devices into previously inaccessible territories is stunning. We establish stationary identification devices (surveillance cameras, weather sensors, industrial meters etc.) and launch movable devices (flying drones, swimming drones, balloons, UAVs, self-driving cars, rovers etc.) Check out excellent hardware trends for 2015. Today we put plenty of remora devices onto the cars and ourselves. Further miniaturization will allow to take devices inside ourselves. The evidence is that Hardware is eating the World.
Wait, there are fabricators or nanofactories, producing hard goods from 3D models! 3D printed goods and 3D printed hamburger are evidences of Software directly eating the World. Then, the conclusion could be that Software is eating the World previously eaten by Hardware, while Hardware is eating the rest of the World at higher pace than Software is eating via fabrication.
Things are not so straightforward. We [you and me] have stuck in silicon world. Those ruminations are true for electrical/digital devices/technologies. Things are not limited to digital and electrical. The movement of biohackers can’t be ignored. Those guys are doing garage bio experiments on 5K equipment exactly as Jobs and Woz did electrical/digital experiments in their garage during PC era birth.
Biohackers are also eating the World. I am not talking about standard boring initiation [of biohacker] to make something glowing… There are amazing achievements. One of them is night vision. Electrical/digital approach to night vision is infra red camera, cooler and analog optical picture into your eyes, or radio scanner and analog/digital reconstruction of the scene for your eyes. Bio approach is injection of Chlorin e6 drops into your eyes. With the aid of Ce6 you could see in the darkness in the range of 10 to 50 meters. Though there is some controversy with that Ce6 experiment.
The new conclusion for the “Eaters Club” is this:
Will convergence of hardware and bio happen as it happened with software and hardware? I bet yes. For remote devices it could be very beneficial to take energy from the ambient environment, which potentially could be implemented via biological mechanisms.
Time for putting it all together and emphasizing onto practical consequences. Small and smaller devices are needed to wrap entire business (machines, people, areas). Many devices needed, 50 billion by 2020. Networking is needed to connect 50 billion devices. Data flow will grow from 50 billion devices and within the network. Data Gravity phenomenon will become more and more observable, when data attracts apps, services and people to itself. Keep reading for details.
Internet of Things is a sweet spot at the intersection of three technological macro trends: Semiconductors, Telecoms and Big Data. All three parts work together, but have different evolution pace. That’s lead to new rules of the ‘common sense’ emerging within IoT.
Remote devices need networking, good networking. And we got an issue, which will only strengthen. The pace of evolution for semiconductors is 60%, while the pace of evolution of networks is 50%. The pace of evolution of storage technology is even faster than 60% annually. It means that newly acquired data will fit into the network less and less in time [less chances for data to get into the Cloud] . It means that more and more data will be left beyond the network [and beyond the Cloud].
Off-the-Cloud data must be handled in-place, at location of acquisition or so. It means huge growth of Embedded Programming. All those small and smaller devices will have to acquire, store, filter, reduce and sync data. It is Embedded Programming with OS, without OS. It is distributed and decentralized programming. It is programming of dynamic mesh networks. It is connectivity from device to device without central tower. It is new kind of the cloud programming, closest to the ground, called Fog. Hence Fog Programming, Fog Computing. Dynamic mesh networks, plenty of DSP, potentially applicable distributed technologies for business logic foundation such as BitTorrent, Telehash, Blockchain. Interesting times in Embedded Programming are coming. This is just Internet of Things Miniaturization phase. Add smart sensing on those P2P connected small and smaller devices in the Fog, and Internet of Things Identification phase will be addressed properly.
We are building new tools that we will use to build our future. We’re doing it through digitization of the World. Everything physical becomes connected and reflected into its digital representation. Don’t overfocus onto Software, think about Hardware. Don’t overfocus onto Hardware, think about Bio. Expected convergence of software-hardware-bio as most stable and eco-friendly foundation for those 50 billion devices by 2020.
Recall Peter Thiel and biz frustrations nowadays. With digitized connected World we will turn from negative direction within current spiral cycle into positive. And of course we will continue with long term acceleration. The future looks exciting.
Music for reading and thinking: from the near future, Blade Runner, Los Angeles 2019
@ 4th Annual Nordic Cloud & Mobile Security Forum in Stockholm
IoT emerges at the interaction of Semiconductors, Telecoms, Big Data and their laws. Moore’s Law for Semiconductors, observed as 60% annual computing power increase. Nielsen’s Law for Telecoms, observed as 50% annual network bandwidth increase; Metcalfe’s Law for networks, observed as value of the network proportional to the squared number of connected nodes (human and machines, many-to-many). Law of Large Numbers is observed as known average probabilities for everything, that you don’t need statistics anymore. On Venn diagram IoT looks smaller than either of those three foundations – Semiconductors, Telecoms and Big Data, but in reality IoT is much bigger, it is digitization and augmentation of our physical world, both in business and lifestyle.
How people recognize IoT? Propably some see only one web, some see another web, others see few webs? There are good known six webs: Near, Hear, Far, Weird, B2B, D2D [aka M2M]. Near is laptop, PC. Hear is smartphone, smartwatch, armband, wristband, chestband, Google Glass, shoes with some electronics. Far is TV, kiosk, projection surface. Weird is voice and gesture interface to Near and Far, with potential new features emerging. B2B is app-to-app or service-to-service. D2D is device-to-device or machine-to-machine.
People used to sit in front of computer, now we sit within big computer. In 3000 days there will be super machine, let’s call it One, according to Kevin Kelly. It’s operating system is web. One identifies and encodes everything. All screens look into One. One can read it all, all data formats from all data sources. To share is to gain, yep, sharing economy. No bits live outside of One. One is us.
Today we are at Identification of everything, especially visually; and Miniaturization of everything, especially with wearables and M2M. High hopes are onto visual identification and recognition. On the one hand, ubiqutous identification is just needed. On the other hand, visual recognition and classification is probably the way to security in IoT. Instead of enforcing tools or rules, there are policies and some control how those policies applied. The rationale is straightforward: technologies change too fast, hence to build something lasting, you should build policies. Policies are empowered by some technology, but remain other technologies agnostic.
Fifth wave is augmentation of life with software and hardware…
Who is IoT today? Let’s take Uber. Today it is not. In several years with self-driven cars it will be. Tim O’Reilly perfectly described IoT as ecosystem of things and humans. Below is comparison, with significantly extended outlook of tomorrow.
It is great step towards personalized experience that Uber linked Spotify to your cab, so that you experience your individual stage in any Uber car. More about personal experience in my previous post Consumerism via IoT, delivered in Munich.
Well, high-level mind-washing stuff is interesting, but is there a canonical architecture for IoT? What could I touch as an engineer? There is reference architecture [revealed several weeks ago by Cisco, Intel and others], consisting of seven layers, shown below:
Notice that upper part is Information Technology, which is non-real-time, and which must be personalized. Lower part is Operational Technology, which is real-time or near-real-time, and which is local and geo-spread. Central part is Cloud-aware, which is IT and it’s centralized with strategic geo-distribution, with data centers for primary internet hubs and user locations.
From infosec point of view, top level is broken, i.e. people are broken. They continue to do stupid things, they are lazy, so it’s not rational to try to improve people. They will drive you crazy with BYOD, BYOA and BYOT (bring your own device/app/technology). It is better to invest into technologies which are secure by design. Each architectural layer has own technological & security standards, reinforced by industry standards. Really? Yes for upper part and not obvious for the lower…
Pay attention to the lower part, from Edge Computing and downstairs. It is blurred technology as for today, it could be called Fog. Anyway, Cisco calls it Fog. The Fog perfectly reflects the closest cloud to the ground; encapsulates plenty of computing, storage and networking functionality within. Fog provides localization and context awareness with low latency. Cloud provides global centralization, probably with some latency and less context. Experience on top of Cloud & Fog should provide profiling and personalization, personal UX. The World is flat. The World is not flat. It’s depends on which layer of IoT you are now.
Data growths too fast, that in many scenarios it simply can’t be moved to the Cloud for intelligence; hence BI comes to the Data. Big Data has big gravity and it attracts apps, services to itself. And hackers too. Gathering, filtering, normalizing, accumulating data at location or elsewhere, outside the cloud, is called Edge Computing. It is often embedded programming of single-card computers or other mediums (controllers, Arduino, Raspberry Pi, Tessel.io, smartphones when much computing power required).
Fog Computing is a virtualized distributed platform that provides computing, storage, and networking services between devices and the cloud. Fog Computing is widespread, uncommon, interconnected. Fog Computing is location-aware, real-time/near-real-time, geo-spread, large-scale, multi-node, heterogeneous. Check out http://www.slideshare.net/MichaelEnescu/michael-enescu-cloud-io-t-at-ieee
Fog is hot for infosec, because plenty of logic and data will sit outside of the cloud, outside of the office, somewhere in the field… so vulnerable because of immaturity of IoT technologies at that low level.
How to find or build technologies for the Fog Computing, which would be secure by design? Which would live quite long, like TCP/IP:) Is it possible? Are some candidate technologies exist so far? And potentially they should be built on top of proven open-sourced tools & technologies, to keep trust and credibility. It all must synergize at large collaboration scale to breakthrough with proper tech fabric. So what do we have today? Fog is about computing, storage and networking, just a bit different from the same stuff in the cloud or in the office.
Computing. Which computing is secure, transactional and distributed? And could fit onto Raspberry Pi? Ever thought about Bitcoin? Ha! Bitcoin’s Block Chain algorithm is exactly the secure transactional distributed engine, even platform. Instead of computing numbers for encryptions and mine Bitcoins, you could do more useful computing job. Technology has all necessary features included in it by design. Temporary and secure relations are established between smartphones and gadgets, devices and transactions happen. Check out Block Chain details.
Storage. Data sending & receiving. Which technology is distributed, efficient of low-bandwidth networks, reliable and proven? BitTorrent! BitTorrent is not for pirates, it is for Fog Computing. For mesh networks and efficient data exchange on many-to-many topologies, built over P2P protocol. BitTorrent is good for video streaming too. Check out BitTorrent details .
Identification. Well, may be it’s not identification of everything and everyone, but authentication and authorization is needed anyway, and needed right now. Do we have such technology? Yes, it is Telehash! Good for mesh networks, based on JSON, enables secure messaging. Check out Telehash details.
Fog Computing is new field, we have to use applicable secure technologies there, or create new better technologies. Looks like it is going to be hybrid, something applied, something invented. Check out original idea from IBM Research for original arguments and ideation.
A proposal is to go ahead with OWASP Top 10 for IoT. Just google for OWASP and code like I10 or I8. You will get the page with recommendations how to secure certain aspect of IoT. The list of ten doesn’t match seven layers of reference architecture precisely, while some relevance is obvious. Some layers are matched. Some security recommendations are cross-functional, e.g. Privacy.
For Fog Computing pay attention to I2, I3, I4, I7, I9, I10. All those recommendations could be googled by those names; though they are slightly different at OWASP site. Below is a list of hyperlinks for your convenience. Enjoy!
I1 Insecure Web Interface
I2 Insufficient Authentication/Authorization
I3 Insecure Network Services
I4 Lack of Transport Encryption
I5 Privacy Concerns
I6 Insecure Cloud Interface
I7 Insecure Mobile Interface
I8 Insufficient Security Configurability
I9 Insecure Software/Firmware Updates
I10 Poor Physical Security
More about Internet of Things, especially from user point of view could be found at my recent post Consumerism via IoT.
We buy things we don’t need
with money we don’t have
to impress people we don’t like
That sucks. That got to be changed. Fight Club changed it that violent way… Thanks God it was in book/movie only. We are changing it different way, peacefully, via consumerism. We are powerful consumers in new economy – Experience Economy. Consumers don’t need goods only, they need experiences, staged from goods, services and something else.
Staging experience is difficult. Staging personal experience is a challenge for this decade. We have to gather, calculate and predict about literally each customer. The situation gets more complicated with growing Do It Yourself attitude from consumers. They want to make it, not just to buy it…
If you have not so many customers then staging of experience could be done by people, e.g. Vitsoe. They are writing on letter cards exclusively for you! To establish realistic human-human interface from the very beginning. You, as consumer, do make it, by shooting pictures of your rooms and describing the concept of your shelving system. New Balance sneakers maker directly provides “Make button”, not buy button, for number of custom models. You are involved into the making process, it takes 2 days, you are informed about the facilities [in USA] and so on; though you are just changing colors of the sneaker pieces, not a big deal for a man but the big deal for all consumers.
There are big etalons in Experience Economy to look for: Starbucks, Walt Disney. Hey, old school guys, to increase revenue and profit think of price goes up, and cost too; think of staging great experiences instead of cutting costs.
Computers disrupted our lives, lifestyle, work. Computers changed the world and still continue to change it. Internet transformed our lives tremendously. It was about connected machines, then connected people, then connected everything. The user used to sit in front of computer. Today user sits within big computer [smart house, smart ambient environment, ICU room] and wears tiny computers [wristbands, clasps, pills]. Let’s recall six orders of magnitude for human-machine interaction, as Bill Joy named them – Six Webs – Near, Hear, Far, Weird, B2B, D2D. http://video.mit.edu/embed/9110/
Nowadays we see boost for Hear, Weird, D2D. Reminder what they are: Hear is your smartphone, smartwatch [strange phablets too], wearables; Weird is voice interface [automotive infotaintent, Amazon Echo]; D2D is device to device or machine to machine [aka M2M]. Wearables are good with anatomical digital gadgets while questionable with pseudo-anatomical like Google Glass. Mobile first strategies prevail. Voice interop is available on all new smartphones and cars. M2M is rolling out, connecting “dumb” machines via small agents, which are connected to the cloud with some intelligent services there.
At the end of 2007 we experienced 5000 days of the Web. Check out what Kevin Kelly predicts for next 5000 days [actually less than 3000 days from now]. There will be only One machine, its OS is web, it encodes trillions of things, all screens look into One, no bits live outside, to share is to gain, One reads it all, One is us…
Well, next 3000 days are still to come, but where we are today? At two slightly overlapping stages: Identification of Everything and Miniaturization & Connecting of Everything. Identification difficulties delay connectivity of more things. Especially difficult is visual identification. Deep Neural Networks did not solve the problem, reached about 80% accuracy. It’s better than old perceptrons but not sufficient for wide generic application. Combinations with other approaches, such as Random Forests bring hope to higher accuracy of visual recognition.
Huge problem with neural networks is training. While breakthrough is needed for ad hoc recognition via creepy web camera. Intel released software library for computer vision OpenCV to engage community to innovate. Then most useful features are observed, improved and transferred from sw library into hw chips by Intel. Sooner or later they are going to ship small chips [for smartphones for sure] with ready-made special object recognition bits processing, so that users could identify objects via small phone camera in disconnected mode with better accuracy than 85-90%, which is less or more applicable for business cases.
As soon as those two IoT stages [Identification and Miniaturization] are passed, we will have ubiquitous identification of everything and everyone, and everything and everyone will be digitized and connected – in other words we will create a digital copy of us and our world. It is going to be completed somewhere by 2020-2025.
Then we will augment ourselves and our world. Then I don’t know how it will unfold… My personal vision is that humanity was a foundation for other more intelligent and capable species to complete old human dream of reverse engineering of this world. It’s interesting what will start to evolve after 2030-2040. You could think about Singularity. Phase shift.
Well, back to today. Today we are still comfortable on the Earth and we are doing business and looking for lucrative industries. Which industries are ready to pilot and rollout IoT opportunities right away? Here is a list by Morgan Stanley since April 2014:
Utilities (smart metering and distribution)
Insurance (behavior tracking and biometrics)
Capital Goods (factory automation, autonomous mining)
Agriculture (yield improvement)
Pharma (critical trial monitoring)
Healthcare (leveraging human capital and clinical trials)
Medtech (patient monitoring)
Automotive (safety, autonomous driving)
Time to draw baseline. Everybody is sure to have true understanding of IoT. But usually people have biased models… Let’s figure out what IoT really is. IoT is synergistic phenomenon. It emerged at the interaction of Semiconductors, Telecoms and Software. There was tremendous acceleration with chips and their computing power. Moore’s Law still has not reached its limit [neither at molecular nor atomic level nor economic]. There was huge synergy from wide spread connectivity. It’s Metcalfe’s Law, and it’s still in place, initially for people, now for machines too. Software scaled globally [for entire planet, for all 7 billions of people], got Big Data and reached Law of Large Numbers.
As a result of accelerated evolution of those three domains – we created capability to go even further – to create Internet of Things at their intersection, and to try to benefit from it.
If global and economic description is high-level for you, then here you go – 7 levels of IoT – called IoT Reference Architecture by Cisco, Intel and IBM in October 2014 at IoT World Forum. A canonical model sounds like this: devices send/receive data, interacting with network where the data is transmitted, normalized and filtered using edge computing before landing in databases/data storage, accessible by applications and services, which process it [data] and provide it to people, who will act and collaborate.
You could ask which company is IoT one. This is very useful question, because your next question could be about criteria, classifier for IoT and non-IoT. Let me ask you first: is Uber IoT or not?
Today Uber is not, but as soon as the cars are self-driven Uber will be. An only missing piece is a direct connection to the car. Check out recent essay by Tim O’Reilly. Another important aspect is to mention society, as a whole and each individual, so it is not Internet of Things, but it is Internet of Things & Humans. Check out those ruminations http://radar.oreilly.com/2014/04/ioth-the-internet-of-things-and-humans.html
Humans are consumers, just a reminder. Humans is integral part of IoT, we are creating IoT ourselves, especially via networks, from wide social to niche professional ones.
Chips and networks are good, let’s look at booming software, because technological process is depending vastly on software now, and it’s accelerating. Each industry opens more and more software engineering jobs. It started from office automation, then all those classical enterprise suites PLM, ERP, SCADA, CRM, SCM etc. Then everyone built web site, then added customer portal, web store, mobile apps. Then integrated with others, as business app to business app aka B2B. Then logged huge clickstreams and other logs such as search, mobile data. Now everybody is massaging the data to distill more information how to meet business goals, including consumerism shaped goals.
So whatever your industry is, think about more software coding and data massage. Plenty of data, global scale, 7 billions of people and 30 billions of internet devices. Think of traditional and novel data, augmented reality and augmented virtuality are also digitizers of our lives towards real virtuality.
If you know how, then don’t read further, just go ahead with your vision, I will learn from you. For others my advice will be to design for personal experience. Just continue to ride the wave of more & more software piece in the industries, and handle new software problems to deliver personal experience to consumers.
First of all, start recognizing novel data sources, such as Search, Social, Crowdsourced, Machine. It is different from Traditional CRM, ERP data. Record data from them, filter noise, recognize motifs, find intelligence origins, build data intelligence, bind to existing business intelligence models to improve them. Check out Five Sources of Big Data.
Second, build information graphs, such as Interest, Intention, Consumption, Mobile, Social, Knowledge. Consumer has her interests, why not count on them? Despite the interests consumer’s intentions could be different, why not count on them? Despite the intentions consumer’s consumption could be different, why not count on them? And so on. Build mobility graph, communication graph and other specific graphs for your industry. Try to build a knowledge graph around every individual. Then use it to meet that’s individual expectations or bring individualized unexpected innovations to her. Check out Six Graphs of Big Data.
As soon as you grasp this, your next problem will be handling of multi-modality. Make sure you got mathematicians into your software engineering teams, because the problem is not trivial, exactly vice versa. Good that for each industry some graph may prevail, hence everything else could be converted into the attributes attached to the primary graph.
Taking simplified PLM as BEFORE –> DURING –> AFTER…
Design of the product should start as early as possible, and it is not isolated, instead foster co-creation and co-invention with your customers. There is no secret number how much of your IP to share publicly, but the criteria is simple – if you share insufficiently, then you will not reach critical mass to trigger consumer interest to it; and if you share too much, your competitors could take it all. The rule of thumb is about technological innovativeness. If you are very innovative, let’s say leader, then you could share less. Examples of technologically innovative businesses are Google, Apple. If you are technologically not so innovative then you might need to share more.
The production or assembly should be as optimal as possible. It’s all about transaction optimization via new ways of doing the same things. Here you could think about Coase Law upside down – outsource to external patterns, don’t try to do everything in-house. Shrink until internal transaction cost equals to external. Specialization of work brings [external] costs down. Your organization structure should reduce while the network of partners should grow. In the modern Internet the cost of external transactions could be significantly lower than the cost of your same internal transactions, while the quality remains high, up to the standards. It’s known phenomenon of outsourcing. Just Coase upside down, as Eric Schmidt mentioned recently.
Think about individual customization. There could be mass customization too, by segments of consumers… but it’s not so exciting as individual. Even if it is such simple selection of the colors for your phones or sneakers or furniture or car trim. It should take place as late as possible, because it’s difficult to forecast far ahead with high confidence. So try to squeeze useful information from your data graphs as closer to the production/assembly/customization moment as possible, to be sure you made as adequate decisions as could be made at that time. Optimize inventory and supply chains to have right parts for customized products.
Then try to keep the customer within experience you created. Customers will return to you to repeat the experience. You should not sit and wait while customer comes back. Instead you need to evolve the experience, think about ecosystem. Invent more, costs may raise, but the price will raise even more, so don’t push onto cost reduction, instead push onto innovativeness towards better personal experiences. We all live within experiences [BTW more and more digitized products, services and experiences]. The more consumer stays within ecosystem, the more she pays. It’s experience economy now, and it’s powered by Internet of Things. May be it will rock… and we will avoid Fight Club.
This post is related to the published visuals from my Internet of Everything session at THINGS EXPO in June 2014 in New York City. The story is relevant to the visuals but there is no firm affinity to particular imagery. Now there story is more like a stand alone entity.
Guess how many things (devices & humans) are connected to the Internet? Guess who knows? The one who produces those routers, that moves your IP packets across the global web – Cisco. Just navigate to the link http://newsroom.cisco.com/ioe and check the counter in right top corner. The counter doesn’t look beautiful, but it’s live, it works, and I hope will continue to work and report approximate number of the connected things within the Internet. Cisco predicts that by 2020, the Internet of Everything has the potential to connect 50 billion. You could check yourself whether the counter is already tuned to show 50,000,000,000 on 1st of January 2020…
Old good globalization was already described in The World is Flat. With the rise of smart phones with local sensors (GPS, Bluetooth, Wi-Fi) the flatness of the world has been challenged. Locality unflattened the world. New business models emerged as “a power of local”. The picture got mixed: on one hand we see same burgers, Coca-Cola and blue jeans everywhere, consumed by many; while on the other hand we already consume services tailored to locality. Even hard goods are tailored to locality, such as cars for Alaska vs. Florida. Furthermore, McDonald’s proposes locally augmented/extended menu, and Coca-Cola wraps the bottles with locally meaningful images.
Location itself is insufficient for the next big shift in biz and lives. A context is a breakthrough personalizer. And that personal experience is achievable via more & smaller electronics, broadband networks without roaming burden, and analytics from Big Data. New globalization is all about personal experience, everywhere for everyone.
Today you have to take your commodities, together with made good, together with services, bring it all onto the stage and stage personal experience for a client. It is called Experience Economy. Nowadays clients/users want experiences. Repeatable experiences like in Starbucks or lobster restaurant or soccer stadium or taxi ride. I already have a post on Transformation of Consumption. Healthcare industry is one of early adopters of the IoT, hence they deserved separate mentioning, there is a post on Next Five Years of Healthcare.
So you have to get prepared for the higher prices… It is a cost of staging of personal experience. Very differentiated offering at premium price. That’s the economical evolution. Just stick to it and think how to fit there with your stuff. Augment business models correspondingly. Allocate hundreds of MB (or soon GB) for user profiles. You will need a lot to store about everybody to be able to personalize.
Remember that it’s not all about consumer. There are many things around consumer. They are part of the context, service, ecosystem. Count on them as well. People use those helper things [machines, software] to improve something in biz process or in life style, either cost or quality or time or emotions. Whatever it is, the interaction between people and between people-machine is crucial for proper abstraction and design for the new economy.
Creator of Berkeley Unix, creator of vi editor, co-founder of Sun Microsystems, partner at KPCB – Bill Joy – outlined six levels of human-human, human-machine, machine-machine interaction. That was about 20 years ago.
About 10 years ago Bill Joy reiterated on Six Webs. He pointed to “The Hear Web” as most promising and exciting for innovations.
The human body is anatomically the same through the hundreds of years. Hence the ergonomics of wearables and handhelds is predefined. Bracelets, wristwatches, armbands are those gadgets that could we wear for now on our arms. The difference is in technology. Earlier it was mechanical, now it is electrical.
We are still not there with human augmentation to speak about underskin chips… but that stuff is being tested already… on dogs & cats. Microchip with information about rabies vaccination is put under the skin. Humans also pioneer some things, but it is still not mainstream to talk much about.
For sure “The Hear Web” was a breakthrough during recent years. The evolution of smartphones was amazing. The emergence of wrist-sized gadgets was pleasant. We are still to get clarity what will happen with glasses. Google experiments a lot, but there is a long way to go until the gadget is polished. That’s why Google experiments with contact lenses. Because GLASS still looks awkward…
The brick design of touch smartphone is not the true final one. I’ve figured out significant issue with iPhone design. LG Flex is experimenting with bendable, but that’s probably not significantly better option. High hopes are on Graphene. Nokia sold it’s plastic phone business to Microsoft, because Nokia got multi-billion grant to research in graphene wearables. Graphene is good for electricity, highly durable, flexible, transparent. It is much better for the new generation of anatomically friendly wearables.
D2D stands for Device-to-Device. There must be standards. High hopes are on Qualcomm. They are respected chipset & patents maker. They propose AllJoyn – open source approach for connecting things – during recent years. All common functionality such as discovery/onboarding, Wi-Fi comms, data streaming to be standardized and adopted by developers community.
AllSeen Alliance is an organization of supporters of the open source initiative for IoT. It is good to see there names like LG, Sharp, Haier, Panasonic, Technicolor (Thomson) as premier members, Cisco, Symantec and HTC as community members. And really nice to see one of etalons of Wikinomics – Local Motors!
For sure Google would try to push Android onto as many devices as possible, but Google must understand that they are players in plastic gadgets. It’s better to invest money into hw & graphene companies and support the alliance with money and authority. IoT needs standards, especially at D2D/M2M level.
If you know how – then go ahead. Else – design for personal experience. Internet of Everything includes semiconductors, telecoms and analytics from Big Data.
Assuming you are in software business, let semiconductors continue with Moore’s Law, let telecoms continue with Metcalfe’s Law, while concentrate on Big Data to unlock analytics potential, for context handling, for staging personal experience. Just consider that Metcalfe’s Law could be spread onto human users and machines/devices.
Start design of Six Graphs of Big Data from Five Sources of Big Data. The relation between graphs and sources is many-to-many. Blending of the graphs is not trivial. Look into Big Data Graphs Revisited. Conceptualization of the analytics pipeline is available in Advanced Analytics, Part I. Most interesting graphs are Intention & Consumption, because first is a plan, second is a fact. When they begin to match, then your solution begin to rock. Write down and follow it – the data is the next currency. 23andme and Uber logs so much data besides the cap of service you see and consume…
There are clear five waves of the IoT. Some of those waves overlap. Especially ubiquitous identification of people or things indoors and outdoors. If the objects is big enough to be labeled with RFID tag or visual barcode than it is easy. But small objects are not labeled neither with radio chip nor with optical code. No radio chip because it is not good money-wise. E.g. bottles/cans of beer are not labeled because it’s too expensive per item. The pallets of beer bottles are labeled for sure, while single bottle is not. There is no optical code as well, to not spoil the design/brand of the label. Hence it is a problem to look for alternative identification – optical – via image recognition.
Third wave includes image recognition, which is not new, but it is still tough today. Google has trained Street View brain to recognize house numbers and car plates at such high level of accuracy, that they could crack captcha now. But you are not Google and you will get 75-78% with OpenCV (properly tuned) and 79-80% with deep neural networks (if trained properly). The training set for deep learning is a PITA. You will need to go to each store & kiosk and make pictures of the beer bottles under different light conditions, rotations, distances etc. Some footage could be made in the lab (like Amazon shoots the products from 360) but plenty of work is your own.
Fourth wave is about total digitization of the world, then newer world will work with digital things vial telepresence & teleoperations. Hopefully we will dispense with all those power adapters and wires by that time. “Software is eating the World”. All companies become software companies. Probably you are comfortable with digital music (both consuming and authoring), digital publishing, digital photos and digital movies. But you could have concerns with digital goods, when you pay for the 3D model and print on 3D printer. While atomic structure of the printed goods is different, your concern is right, but as soon as atomic structure is identical [or even better] then old original good has, then your concern is useless. Read more in Transformation of Consumption.
With 3D printing of hard goods it’s less or more understandable. Let’s switch tp 3D printed food. Modern Meadow printed a burger year ago. It costed $300K, approximately as much as Sergei Brin (Googler) invested into Modern Meadow. Surprised? Think about printed newest or personal vaccines and so forth…
Is Uber IoT or not? With human drivers it is not. When human-driven cabs are substituted by self-driving cabs, then Uber will become an IoT. There is excellent post by Tim O’Reilly about Internet of Things & Humans. CEO of Box.com Levie tweeted “Uber is a $3.5 billion lesson in building for how the world *should* work instead of optimizing for how the world *does* work.” IoT is not just more data [though RedHat said it is], IoT is how this world should work.
It was vision for 5000 days, but today only 3000 days left. Check it out.
Check out There will be no End of the World. We will build so big and smart web, that we as humans will prepare the world to the phase shift. Our minds are limited. Our bodies are weird. They survive in very narrow temperature range. They afraid of radiation, gravity. We will not be able to go into deep space, to continue reverse engineering of this World. But we are capable to create the foundation for smarter intelligence, who could get there and figure it out. Probably we would even don’t grasp what it was… But today IoT pathway brings better experiences, more value, more money and more emotions.
Let’s check Cisco internet of things counter. ~300,000 new things have connected to the Internet while I wrote this story.
Some time ago I’ve outlined Six Graphs of Big Data as a pathway to the individual user experience. Then I’ve did the same for Five Sources of Big Data. But what’s between them remained untold. Today I am going to give my vision how different data sources allow to build different data graphs. To make it less dependent on those older posts, let’s start from the real-life situation, business needs, then bind to data streams and data graphs.
Same data in different contexts has different value. When you are late to the flight, and you got message your flight was delayed, then it is valuable. In comparison to receiving same message two days ahead, when you are not late at all. Such message might be useless if you are not traveling, but airline company has your contacts and sends such message on the flight you don’t care about. There was only one dimension – time to flight. That was friendly description of the context, to warm you up.
Some professional contexts are difficult to grasp by the unprepared. Let’s take situation from the office of some corporation. Some department manager intensified his email communication with CFO, started to use a phone more frequently (also calling CFO, and other department managers), went to CFO office multiple times, skipped few lunches during a day, remained at work till 10PM several days. Here we got multiple dimensions (five), which could be analyzed together to define the context. Most probably that department manager and CFO were doing some budgeting: planning or analysis/reporting. Knowing that, it is possible to build and deliver individual prescriptive analytics to the department manager, focused and helping to handle budget. Even if that department has other escalated issues, such as release schedule or so. But severity of the budgeting is much higher right away, hence the context belongs to the budgeting for now.
By having data streams for each dimension we are capable to build run-time individual/personal context. Data streams for that department manager were kind of time series, events with attributes. Email is a dimension we are tracking; peers, timestamps, type of the letter, size of the letter, types and number of attachments are attributes. Phone is a dimension; names, times, durations, number of people etc. are attributes. Location is a dimension; own office, CFO’s office, lunch place, timestamps, durations, sequence are attributes. And so on. We defined potentially useful data streams. It is possible to build an exclusive context out of them, from their dynamics and patterns. That was more complicated description of the context.
Well, well, but how to interpret those data streams, how to interpret the context? What we have: multiple data streams. What we need: identify the run-time context. So, the pipeline is straightforward.
First, we have to log the Data, from each interested dimension. It could be done via software or hardware sensors. Software sensors are usually plugins, but could be more sophisticated, such as object recognition from surveillance cameras. Hardware sensors are GPS, Wi-Fi, turnstiles. There could be combinations, like check-in somewhere. So, think that it could be done a lot with software sensors. For the department manager case, it’s plugin to Exchange Server or Outlook to listen to emails, plugin to ATS to listen to the phone calls and so on.
Second, it’s time for low-level analysis of the data. It’s Statistics, then Data Science. Brute force to ensure what is credible or not, then looking for the emerging patterns. Bottleneck with Data Science is a human factor. Somebody has to look at the patterns to decrease false positives or false negatives. This step is more about discovery, probing and trying to prepare foundation to more intelligent next step. More or less everything clear with this step. Businesses already started to bring up their data science teams, but they still don’t have enough data for the science:)
Third, it’s Data Intelligence. As MS said some time ago “Data Intelligence is creating the path from data to information to knowledge”. This should be described in more details, to avoid ambiguity. From Technopedia: “Data intelligence is the analysis of various forms of data in such a way that it can be used by companies to expand their services or investments. Data intelligence can also refer to companies’ use of internal data to analyze their own operations or workforce to make better decisions in the future. Business performance, data mining, online analytics, and event processing are all types of data that companies gather and use for data intelligence purposes.” Some data models need to be designed, calibrated and used at this level. Those models should work almost in real-time.
Fourth, is Business Intelligence. Probably the first step familiar to the reader:) But we look further here: past data and real-time data meet together. Past data is individual for business entity. Real-time data is individual for the person. Of course there could be something in the middle. Go find comparison between stats, data science, business intelligence.
Fifth, finally it is Analytics. Here we are within individual context for the person. There worth to be a snapshot of ‘AS-IS’ and recommendations of ‘TODO’, if the individual wants, there should be reasoning ‘WHY’ and ‘HOW’. I have described it in details in previous posts. Final destination is the individual context. I’ve described it in the series of Advanced Analytics posts, link for Part I.
Data streams come from data sources. Same source could produce multiple streams. Some ideas below, the list is unordered. Remember that special Data Intelligence must be put on top of the data from those streams.
In-door positioning via Wi-Fi hotspots contributing to mobile/mobility/motion data stream. Where the person spent most time (at working place, in meeting rooms, on the kitchen, in the smoking room), when the person changed location frequently, directions, durations and sequence etc.
Corporate communication via email, phone, chat, meeting rooms, peer to peer, source control, process tools, productivity tools. It all makes sense for analysis, e.g. because at the time of release there should be no creation of new user stories. Or the volumes and frequency of check-ins to source control…
Biometric wearable gadgets like BodyMedia to log intensity of mental (or physical) work. If there is low calories burn during long bad meetings, then that could be revealed. If there is not enough physical workload, then for the sake of better emotional productivity, it could be suggested to take a walk.
Ok, but how to build something tangible from all those data streams? The relation between Data Graphs and Data Streams is many to many. Look, it is possible to build Mobile Graph from the very different data sources, such as face recognition from the camera, authentication at the access point, IP address, GPS, Wi-Fi, Bluetooth, check-in, post etc. Hence when designing the data streams for some graph, you should think about one to many relations. One graph can use multiple data streams from corresponding data sources.
To bring more clarity into relations between graphs and streams, here is another example: Intention Graph. How could we build Intention Graph? The intentions of somebody could be totally different in different contexts. Is it week day or weekend? Is person static in the office or driving the car? Who are those peers that the person communicates a lot recently? What is a type of communication? What is a time of the day? What are person’s interests? What were previous intentions? As you see there could be data logged from machines, devices, comms, people, profiles etc. As a result we will build the Intention Graph and will be able to predict or prescribe what to do next.
Finally, having multiple data graphs we could work on the individual context, personal UX. Technically, it is hardly possible to deal with all those graphs easily. It’s not possible to overlay two graphs. It is called modality (as one PhD taught me). Hence you must split and work with single modality. Select which graph is most important for your needs, use it as skeleton. Convert relations from other graphs into other things, which you could apply to the primary graph. Build intelligence model for single modality graph with plenty of attributes from other graphs. Obtain personal/individual UX at the end.