Dr. Jensen HUANG, the Founder and CEO of NVIDIA in a recent interaction at The Hong Kong University of Science and Technology (HKUST) shared his views on the groundbreaking innovations in AI and the transformative power of collaboration, technology, leadership, entrepreneurship and more.
The transformative and groundbreaking capability happens when an artificial intelligence network can learn to understand data of all kinds, which include language and images and sequences of proteins and sequences of amino acids and sequences of chemicals. And all of a sudden, we now have computers that can understand the meaning of words. And due to generative AI, we can translate from one modality of information to another modality of information, for instance, from text to images, from text to text, from protein to text, from text to protein, and from text to chemicals, this universal, initially a universal function approximator, evolved into a universal language translator of every kind. Therefore, the question is, what can we do with that? Furthermore, you see the number of startups around the world and several capabilities with the combination of all these different modalities and capabilities. Hence, I think an amazing breakthrough is that we can now understand the meaning of information – incredibly difficult information.
Also, what does that mean to you if you are a digital biologist? So that you can understand the meaning of the data you are looking at so that you can find a needle in a haystack. What does that mean if you are - in the case of NVIDIA, a chip designer, or system designer? What does that mean to you if you are in agtech or climate science or climate tech looking for a new material? Thus, this is the groundbreaking fact is that we now have the concept of a universal translator. It helps you in understanding even the most difficult topics.
From the perspective of computer science, we have reinvented the whole stack. When you think about computer science, you have to think about software development, how software is developed. We used to code software manually. We imagine what function it is that we would like to implement, what is the algorithm that we would like to implement, and we use our own creativity and we type it into the computer.
I started with FORTRAN, and I learned Pascal, C and C++, and, each one of these languages allows us to express our thoughts into code, and that code runs great on CPUs. All of a sudden, now we use observed data, and we input this observed data to a computer, and we instruct about the function that we see inside this code. We also look at the patterns and relationships we observe by studying all the data we presented to you. Rather than using code and coding, we now use machine learning, and the machine does not generate software, but it generates neural networks that are processed on GPUs.
Thus, from coding to machine learning, from CPUs to GPUs, and since GPUs are much more powerful, the type of software we can now develop is extraordinary, and what sits on top of it is artificial intelligence. That is what has emerged. Therefore, computer science has been transformed much.
Now, the question is, what happens to our industry? We are all racing to use machine learning to discover new AIs, and what is AI? Maybe that is one of the factors about AI that you know very well is the automation of cognition, automation of problem-solving and Problem-solving could be distilled down to three basic ideas that you observe and perceive the environment, understand it, reason about it, and then come up with a plan to interact with it, whatever you decide your goals are and so perception, reasoning, and planning, the three fundamental steps of problem-solving.
Perception, reasoning, and planning could be broken down into perceiving the environment around your car, reasoning about the location that you are, and the location of all the other cars around you, planning how to drive. Therefore, I described about self-driving cars. These self-driving cars in one manifestation, would be called a digital chauffeur.
Another instance is where we can observe a CT scan, understand it, reason about everything that we see, and come to the conclusion there might be some anomaly that might be a tumor or other disease, and then we may decide to highlight it and describe it to the radiologist. Now, you are a digital radiologist. In almost everything that we do, we can come up with some expression that artificial intelligence could then perform a particular task.
Thus what happens is, if we have enough of those digital agents and those digital agents are interacting with the computer that is generating digital artificial intelligence, - the total consumption of all of us into a data center makes the data center look like it's producing tokens, or otherwise known as digital intelligence. Therefore, today let me now describe it a little differently. A hundred years ago, General Electric and Westinghouse came up with a new type of instrument - a new type of machine, it was called a dynamo, and eventually became an AC generator and they were smart to invent a consumer, a consumption of the electricity that they were able to produce and that consumption, would be inventions such as the light bulbs and toasters. They created all kinds of digital appliances or electrical appliances that consumes the electricity that these plants would produce. Let us look at what we are doing now. We are creating co-pilots and ChatGPT.
We are creating all these different intelligence, basically light bulbs and toasters. Think of them as appliances that all of us would use, but you would connect it to a factory. It used to be an AC power generation factory, but this new factory is a digital intelligence factory. And so what is just, from an industrial perspective, really what's happening here is we are now creating a new industry, and this new industry takes energy in and produces digital intelligence. And the digital intelligence would be used by all kinds of different applications and its consumption we believe, is going to be quite large and this entire industry never existed before, just like the AC generation industry never existed before that.
Moore's law depended on two concepts. One concept was VLSI scaling, and that was because of Carver Mead and the textbook by Mead and Conway really inspired my generation. The second is Dennard scaling. Constant current density scaling of transistors coupled with the shrinking of the transistors made it possible for us to double the performance. If you will, double the performance of semiconductors every couple of years or so every one and a half years, so that would be 10 times every five years, 100 times every 10 years. And the other, what we are experiencing now is that the larger your neural network can become, and the more data that you train that neural network with, the more intelligent the AI seems to become.
It is an empirical law, just like Moore's law versus what we call that the scaling law, and the scaling law appears to be continuing. But the one factor that we also know about intelligence is that pre-training, just taking all of the data in the world and discovering knowledge from it automatically, pre-training is not enough. Although going to college and graduating from college is a very important milestone, it's not enough. There is post-training, which is learning a particular skill very deeply. And post-training requires reinforcement learning human feedback, reinforcement learning AI feedback, synthetic data generation, multi-path learning, and reinforcement learning.
There are a myriad of techniques, but basically, you are now going deep into a particular domain, and you are trying to learn something very deep about it. That's post-training. When you select a particular career, you are going to learn a lot again. And then post that, it's called thinking and we call it test time scaling. Some things you just know the answer to.
You have to break the problem down into step-by-step processes, into its first principled elements, and from first principles you have to try to find a solution for each one of them. It may require you to iterate, and simulate various outcomes because the answer is not predictive, and so forth. Thus, we call that thinking and the longer you think, maybe the higher quality the answer would become.
Hence, notice that we now have three areas of artificial intelligence development, where a great deal of computation would result in higher-quality answers. Today, the answers that we have are the best that we can provide. But we need to get to a point where the answer that you get is not the best that we can provide.
And, you still have to decide whether is this hallucinated or not hallucinated? Does this make sense? Is it sensible or not sensible? We have to get to a point where the answer that you get, you largely trust and hence I think that we are several years away from being able to do that.
In the meantime, we have to keep increasing our computation. In the last 10 years, we increased the performance by a million times. What have we really done? What NVIDIA has contributed is that we have taken the marginal cost of computing and we reduced it by a million times. Imagine if there is something in the world that you rely on. It could be electricity. It could be airline ticket. It could be anything you choose. We reduced it in the last 10 years by 1 million times. Well, when something happens, when something is reduced, when the cost of something reduces by a million times, your habits fundamentally change.
How you think about computing fundamentally changed. That is the single greatest contribution NVIDIA ever made. That we made it so that using a machine to go learn an exhaustively enormous amount of data is something that researchers would not even think twice to do. That is why machine learning has taken off.
We use cookies to ensure you get the best experience on our website. Read more...
.jpg "Technology in Cancer Care: A story of evolving paradigms")
.jpg "Bringing Shop Floor to Manufacturer's palm: Smart Factory Solutions for tomorrow")
