Billionaire Investor Vinod Khosla Speaks Out On AI’s Future and the COVID-19 Economy 

Vinod Khosla, a co-founder of the former Sun Microsystems and a longtime technology entrepreneur, venture capitalist and IT sage, makes billions of dollars betting on new technologies.

Khosla shared some of his technology and investment thoughts at a recent tech conference about the future of AI in business, AI chip design and quantum computing -- and even gave some advice to AI developers and companies about how they can successfully navigate the tumultuous times of the COVID-19 pandemic. Khosla gave his remarks at an “Ask Me Anything” Industry Luminary Keynote at the virtual AI Hardware Summit earlier in October. The Q&A was hosted by Rene Haas, the president of Arm’s IP products group, and a former executive with AI chipmaker Nvidia.

Khosla, who is ranked #353 on the Forbes 400 2020 list, has a net worth today of $2.6 billion, largely earned through his investment successes in the tech field. He founded his VC firm, Khosla Ventures, in 2004.

Here are edited segments from that 30-minute Q&A, which centered on questions asked by viewers of the virtual conference:

Rene Haas: What has been the most significant technological advancement in AI in the last year or two? And how do you anticipate it is going to change the landscape of business?

Vinod Khosla

Vinod Khosla: What's surprised me the most is bifurcation along two lines – one that argues that deep learning goes all the way, and others who argue that AGI (artificial general intelligence) requires very different kinds [of uses]. My bet is that each will be good at certain functions. Now, I don't worry about AGI. Being a philosopher, I do worry about AI and AGI being used for most valuable economic functions human beings do. That's where the big opportunity is. What surprised me most is there's been great progress in language models and algorithms. But the outsize role of hardware in building models that are much more powerful, trillions of parameters per model, and how effective they can be, has been surprising. I'm somewhat biased because we have large investors in open AI. On the flip side, we are large investors in companies like Vicarious, which are taking that AGI in a very different approach.

Haas: Building on that a little bit, there are a lot of AI hardware startup companies. Some are well funded, some with high burn rates. When you think about competing with the software support ecosystem, like Nvidia has, how can startups really rely on the strength of their architecture alone? What are the kinds of things that you look at it in terms of guidelines for startups in this space?

Khosla: There's many different markets, you have to be clear. There is a training market in the data center. There's an inferencing market in the data center. There's a market for edge devices where the criteria are very different. And then there's this emerging area of what quantum computing might do in hardware. We can talk about any of these, but what's really interesting to me is how much innovation we are seeing. Companies like Nvidia and the big cloud providers, especially Google and others, have very strong efforts.

And probably the thing we've learned in semiconductors, having access to process technology and process nodes that others don't – that’s where the software ecosystem gives them such a large advantage. It's hard for startups to compete. Now, I could be wrong, but we've tended to avoid digital architectures, for the data center or for inferencing. We've looked at a dozen of those and chosen not to jump in. Because there's bigger players with huge software and process and resource advantages. On the analog side, it's a whole different ballgame. We've invested in analog inference. There's been multiple analog efforts. I think some haven't addressed enough of the problem to get a large enough power advantage.

So, the bottom line for a startup, is that to do better than Nvidia or one of the other larger players or cloud providers, then you've got to talk about 20X to 100X advantage in TeraOPS per watt. I think if you're not in the hundred TeraOPS per watt range, it's going to be hard to sustain a large advantage. And I see most digital efforts sort of in this one to 10 TeraOPS per watt power range. So I find the edge much more promising than the data center.

Haas: What about the difficulties of startups or companies trying to enter this field? Much of it is horizontal in nature. Do they need some kind of vertical stack or some tie into the ecosystems? Do the same challenges apply, relative to being a horizontal versus vertical business or do you think there are some different opportunities there?

Khosla: I think there will be classes of algorithms. There's clearly one class of algorithms around deep learning and things like that. The question of how architecture maps to different types of algorithms, and algorithmic approaches, is a little too early to predict, and that will determine what architectures work best.

On the edge, what's clearly going to be important is power efficiency. The really volume markets are under five watts and $5 and a couple of hundred TeraOPS. That's the price point I look at as differentiated enough for edge devices to do a lot of interesting things. Every speaker, every microphone, every sensor. You start to see price points that go from tens of pennies to a few dollars that go into these very high volume devices. I think that would be a different architecture than the stuff in the data center.

In the data center, whether inferencing and training are the same architecture or the same software stack even, I still think it's open for debate. I think in inferencing, cost matters and efficiency matters. In training, especially for the really large algorithms, probably not so much. So, hard to tap.

And then there's this really surprise thing of what quantum computing will do, and what kinds of algorithms that will run. The things we are most interested in is very specialized applications for quantum computing. We have one effort in drug discovery for quantum computing. I think material science with quantum computing is going to be interesting, possibly some financial services products based on quantum computing. So, plenty of these interesting options. I think for a while we'll see more of a bifurcation, but if I were to predict five years from now I think we'll see more unification around the types of algorithms that do certain economic tasks well.

Rene Haas

Haas: Quantum is something that has been written about for a long time and now you're starting to see some things product-wise that are looking a bit more real. As an investor, and looking at private company opportunities around quantum, do you feel like the time is now to start investing in companies that are doing things around the hardware space in quantum? Or do you look at it and say it's still years away from being commercially viable?

Khosla: In the big company world, it's definitely time for the big companies to be investing, and they're investing heavily. But that's Microsoft, Google, IBM and others. There's also a whole slew of startups where the market and products have emerged slower. And whenever things emerge slower especially on the hardware side, the big companies have an advantage because they can catch up. Whenever it takes lots and lots of resources, then the big companies have an advantage. Autonomous driving is the one area where that's mostly true, but not completely true. We've seen some radical innovation out of startups there.

So, it depends on the pace of development of a technology or deployment. I do think the time is very ripe for quantum software applications, specialized applications, to develop. But given how complex quantum is to use, such as the the interface between quantum and the regular computing world, and the full stack of software and how it runs algorithms, I think specialized algorithms will do better there.

Haas: You're obviously involved in AI chip startups. Looking at the last four years of AI chip startups, are you bullish, in general, looking back? And if so, which areas are you most excited about?

Khosla: When there's radical innovation, it's still interesting. We've seen a lot of startups, but I wouldn't say we've seen radical innovation in architectures or performance or power efficiency. And when I say power efficiency, it's really TeraOPS per watt, which is performance per watt that is really the key metric. If you see the kinds of large jumps, like 20X, 50X, 100X, then that's really interesting. Still, there's less room for it in the data center, more room for it in the edge, but every time I say something like this then some really clever person surprises me with a counter-narrative that actually is pretty compelling. So would I say I'm open for architectures? Yes. Radical changes, yes, and I think that will happen, but it's just very hard to predict today. The predictability on where things go is still low on innovation. But I always say, improbables are not unimportant. We just don't know which improbable is important. In the meantime, the traditional digital data center, even the digital edge, will probably belong to the larger players.

I do want to encourage the folks out there trying to build products. When we did the Nextgen product to compete with Intel, we very quickly got to 50% market share of the under $1,000 PC market, where we were competing on an x86 architecture with Intel. So surprises are possible, and people who take specialized approaches in market segments, there can be very interesting innovation to be done.

Haas: How large is the economic opportunity around AI and what do you think drives it?

Khosla: I'm probably more bullish. Whether you call it, AI or AGI, I think this area will be able to do most economically valuable human functions within the next decade. Probably a lot sooner. They will take time, integrating into regular workflows and traditional systems and all that. But the way I look at it, if we can replace human judgment in a task, you're saving far more money than selling a chip or a computer or something. So, if you can replace a security analyst and do their job, or have one security analyst do the job of five security analysts, or have one physician do the job of five physicians, you're saving gobs of money. And then you get to share in the human labor saving, which is where the large opportunities are. That could belong to both these combination software and hardware systems, I think that opportunity is orders of magnitude larger than any estimate I've seen today.

Haas: 2020 has been a very turbulent year. What advice would you give to tech entrepreneurs who are pushing through a recession and the remarkable situation involving the COVID-19 pandemic, while trying to build a product and build a company? What advice would you give to those entrepreneurs?

Khosla: I think the best ideas survive turbulent times. I find recessions are really the times when bigger companies cut back on some of their spending. I haven't seen that happen in this particular area. That's when people with the best ideas or with passion for a particular vision, leave those companies. So, I do see very good startups during turbulent times in general. Now, one has to be just pragmatic and adapt to the times. When money's cheap, you raise lots of money. When money is not cheap or not easily available you spend less, and take more time doing some fundamental work and getting it right. Which by the way is usually a better strategy than raising lots of money.

I do think that there is lots of opportunity. I think they have to adapt to the times and be much more thoughtful, maybe even more radical in their approach. Take larger leaps because you can take more time before you start spending the money to go to market. One of the things to keep in mind with most technologies –
thinking about the technology has huge implications downstream, but takes very little money. It takes very special talent. Then there's the building of the technology. And then there's the selling, and the sales or marketing usually ends up costing the most. Now's a good time to trade off for more compelling product and postpone some of the sales and marketing while the markets are uncertain. You can't afford to spend lots of money on that. So you have to adjust strategy as an entrepreneur and entrepreneurs do that fairly well.

Haas: What is your own investment philosophy, particularly when it comes to tech companies, and how does your overall portfolio, reflect that philosophy?

Khosla: We like the higher-risk, higher-upside things. I find investors generally reduce risk for good reasons, but make the consequences of success relatively inconsequential. I personally prefer larger risk, which is why I like analog right now, and make the consequential success, be it 50X or 100X better than what's available in the digital domain. I do see plenty of those kinds of opportunities still. I am not discouraged. I'm actually quite encouraged about the opportunities in this area. But, entrepreneurs usually find specialized paths to get to that first MVP product, that early traction, and then use it to broaden.

Haas: Model performance has been increasing slowly in the field of AI. Can you share your insights about that?

Khosla: In certain dimensions, I think that's true. When a technology plays out a certain way, it makes rapid progress in the beginning and then starts to peter out. Software models themselves are getting to a level of saturation. The progress on the hardware side, just scaling hardware, has been stunningly valuable as GTC-3 shows. It may give more of an advantage to the large cloud providers – the people who can build, 500,000 CPU, GPU systems. But that's not for everyday use. I think that still needs to be told.

There are alternative approaches that still need to be discovered. I gave you the example of Vicarious, the robotics company we've invested in. Instead of needing 10 million or 100 million cats to recognize a cat, they're saying ‘can we do it from 10 cats?’ So, maybe data becomes a lot less important. And what implications does that have for hardware architectures? It's very clear to me seeing the early results at Vicarious that it is entirely possible for AI systems to learn as rapidly and with as few examples as humans do, if the architecture is different than deep learning.

My bet is different approaches will be very good at different points, and we'll see that kind of specialization of architectures. A long time ago, 25 to 30 years ago, when you looked at Lego blocks, it came in large yellow, white, red, black and blue blocks. And there were three or four types of components. I think that's where software algorithms in AI may be today. Now, you couldn't build the Sydney Opera House out of Lego blocks back then, but then they got all these specialized components. The possibilities explode exponentially, so the combinations allow a lot more flexibility on what can happen, what systems can do. So, it might be we just need different types of algorithms to explore the capability of end-use systems. And that might have large implications for which hardware architectures work.

Hardware scaling may matter in some of these and clever architectures may matter in others. That's why I'm tracking what quantum computing may do for algorithms. Not just your standard quantum computing Shor's algorithm, etc. but real applications like drug discovery or material science. Or could you do better battery material? Those are really interesting now.

Haas: What advice do you have for first time hardware entrepreneurs, with strong architecture ideas, with really smart engineers, who don't really have a track record, and who haven't done this before -- how do you advise them to position themselves to get into this segment?

Khosla: Silicon Valley is very good at recognizing thoughtful, clever people -- they don't have to have a track record. Most successful entrepreneurs don't have track records. So, I wouldn't be afraid of that. I don't think you need a lot of management experience. Building great teams is probably the single piece of advice I give to entrepreneurs. Great and multi-dimensional teams to go after the problem, even if they haven't done it yet. Also, how the cleverness of your architecture isn’t as important as the end results you deliver. Can you deliver that 20X, 50X over what the traditional players will do for your market? I think people underappreciate how much of an advantage you need in your architecture to make it worthwhile to do that startup.

And one more thing. There's a whole lot of tricks both on the models on the software side, on the hardware side. You can do hardware tricks and there's half a dozen which are very common in hardware and half a dozen that are pretty common in software, like reducing the model size. Everybody really gets there. Others have fundamental long-lasting advantages and if you're doing the startup, focus not on the tricks that give you a 5X improvement, because others will catch up to those tricks, either on software or hardware. Instead, focus on what will be the fundamental innovations five years from now, where you'll still have an advantage.