From [GeForce] to artificial intelligence [AI]: Why did [NVIDIA]'s decision about [CUDA] change the company's future?
Introduction: Can a company give up today's profit to build tomorrow's computing world?
A company spends heavily on a capability its customers are not immediately asking for. Adding that capability increases production costs, but customers are unwilling to pay extra for it. The company nevertheless does not retreat. It is not simply trying to sell a new product; it wants to build a computing platform [Computing Platform] on which others can create their own work in the future.
This was the reasoning behind [NVIDIA]'s decision to bring [CUDA] to its [GeForce] graphics processors [GPU]. As Jensen Huang has explained, the decision did not deliver immediate success. Instead, it brought costs, pressure on profits and a long wait. Later, however, it became one of the foundations for the growth of deep learning [Deep Learning].
1. From an accelerator for a narrow task to a broader computing platform [Computing Platform]
Initially, [NVIDIA] was a company that made a particular task run faster. That specialization gave it strength. At the same time, depending on a narrow market could limit its capacity for research and development [Research and Development].
There was a tension here. Specializing too closely in one task could make broader computing work difficult. Trying to do everything could sacrifice the original specialization. Huang's approach was to move gradually between these two positions.
He describes the programmable pixel shader [Programmable Pixel Shader] as the first step. Next came standards-compatible 32-bit floating-point computation [IEEE-compatible FP32] in shaders [Shaders]. This prompted researchers to ask whether a processor designed for graphics could also perform other computations.
Then came [Cg], based on the programming language [C]. That journey eventually led to [CUDA]. It was therefore not a magical solution created in a single day, but a continuing process of connecting one technical capability to the next.
2. Where [CUDA] was taken mattered more than merely creating it
Creating an excellent technology does not automatically make people use it. It needs people who write software [Developers], and large numbers of people able to use the software they write.
The strength of a computing platform [Computing Platform] lies not only in the elegance of its design, but also in how many people already have it. Huang explains this as the importance of the installed base [Installed Base].
He uses the example of the [x86] architecture, which endured despite criticism because of its widespread use. He also observes that some elegantly designed reduced-instruction-set architectures [RISC Architectures] did not achieve their expected success. This does not mean every [RISC] design failed. The point is that technical elegance alone does not guarantee a platform's success.
At the time, [GeForce] was already selling millions of units a year. [NVIDIA] therefore had an established route for bringing the new computing capability to people.
3. [CUDA] in every [GeForce] processor: Building the foundation before users asked for it
The company could have kept [CUDA] only in expensive research machines. Had it done so, however, far fewer people would have been able to access it.
Instead, it decided to bring that capability into widely available personal computers [PC] through [GeForce]. A product bought for gaming also offered an opportunity for research. A student, a university laboratory or a researcher assembling a computer independently could try new computations using equipment they already had.
This should not be understood to mean that [CUDA] ran on every computer. The point is that the [GeForce] installed base [Installed Base] provided a starting foundation that made the capability accessible to more people.
Huang says the effort went beyond supplying equipment: it included visiting universities to teach, writing books and running classes. Building a platform does not end with selling its product. It also requires developing the knowledge and community that can use it.
4. The immediate price: Higher costs and profits under pressure
The economic burden of this decision was substantial. Huang recalls that adding [CUDA] increased the processor's cost by about 50 percent, while the company was operating at a gross margin [Gross Margin] of about 35 percent.
Gamers would not pay extra for computing capabilities they were not using. The company therefore had to absorb the cost. Huang describes it as large enough to swallow the company's gross profit dollars [Gross Profit Dollars] entirely.
These two percentages cannot simply be subtracted to determine the company's financial position. One is based on production cost; the other is a profit margin calculated from revenue. His point is not a simple arithmetic exercise, but the difficult position of sacrificing current earnings for a future platform.
He also roughly recalls that the company's market capitalization [Market Capitalisation] was around US$6 billion to US$8 billion, before falling to about US$1.5 billion. These are estimates from his recollection. They do not establish that [CUDA] alone caused the entire change.
Nevertheless, his observation that the hoped-for payoff took roughly a decade is important. This was not a decision made for the next quarter's earnings.
5. Why did this decision become a foundation for deep learning [Deep Learning]?
A difficult obstacle to new research is the cost and effort of accessing equipment. If people can start a new experiment on equipment they already own, part of that obstacle is reduced.
Distributed through [GeForce], [CUDA] established a foundation on which researchers could experiment in a shared programming environment [Programming Environment]. The knowledge, software and experience created there could support subsequent work.
As deep learning [Deep Learning] grew, suitable tools for its large-scale parallel computation [Parallel Computing], and people who knew how to use them, were already in place. That was an important advantage.
The entire history of deep learning [Deep Learning] does not depend on a single company or tool. But this decision has an important place in making the necessary computing platform [Computing Platform] widely available. That is why Huang emphasizes the idea that [NVIDIA] is the house that [GeForce] built.
6. Building belief before making the technical decision
A leader may see the future clearly. But that clarity alone is not enough if the board, executives, employees, customers and partners do not accept it.
Huang describes his approach as explaining the reasoning step by step. Whenever new information, research progress or an engineering achievement becomes available, he explains to those around him how it connects to the future direction.
The day a major decision is announced should not come as a sudden shock to everyone. Instead, he believes its foundations should already be so well established that people wonder why it took so long.
That kind of long-term belief and patience also lay behind the [CUDA] decision. Alongside choosing the technical path, he had to prepare the company that would bear its cost for the journey.
7. Not a product alone, but a platform partners can build on
[NVIDIA]'s view of itself as a platform company [Platform Company] also matters here. At the center of that approach is making room for other companies to contribute and build together at every layer.
This does not mean all its software is open source [Open Source]. Giving partners opportunities to build together is different from making source code [Source Code] publicly available.
A good platform does not grow through its creator alone. It also grows through the applications built on it by researchers, developers [Developers], companies and educational institutions. The decision about [CUDA] opened the door to that growth.
8. Today's lesson: Speeding up one processor is not enough
In large distributed computing tasks [Distributed Workloads], making only one part faster does not make the entire system faster by the same amount. Amdahl's law [Amdahl's Law] explains this.
Suppose we can improve only half of the time taken by a task. However much we speed up that half, the unchanged half remains. Even if the improved part became so fast that it took no time at all, the whole task could be at most twice as fast.
That is why the graphics processor [GPU], central processor [CPU], networking [Networking], memory [Memory], power [Power] and cooling [Cooling] must be designed to work together. Huang describes this as extremely close joint design [Extreme Co-design].
There is a connection here with the early [CUDA] decision. The capability of an individual tool is not enough. The whole environment in which it is used must develop alongside it.
Conclusion: A decision that created future uses
The importance of introducing [CUDA] was not simply the arrival of a new programming capability. Its deeper significance was that the capability was combined with [GeForce], which already sold in large numbers, and placed in advance into the hands of people who would use it in the future.
Looking only at the immediate cost might have made the decision appear mistaken. But [NVIDIA] understood that a computing platform's [Computing Platform] value also lies in the applications other people will build on it.
Technology, widespread access, education, a developer community [Developer Community] and corporate patience came together to create a new opportunity. That opportunity later connected with the growth of artificial intelligence [AI]. This is the lesson of the [CUDA] decision: tomorrow's major change needs a foundation that many people can start building on today.
Powered By K7