Technology and Innovation Community

 View Only
  • 1.  Semis Update - AI Accelerator Outlook

    Posted 5 days ago

    A quick review of the outlook for the industry including hyperlinks to sources and definitions:

    What are AI Accelerators?

    AI accelerators are specialized computer chips designed to run AI workloads much faster and more efficiently than general-purpose CPUs.  They are optimized for:

    • Large matrix multiplications
    • Parallel processing
    • High memory bandwidth
    • Energy efficiency per calculation
    These operations dominate ML training and and inference, which is why specialized chips outperform traditional processors by large margins. Large Hyperscaler companies that operate massive data centers (Amazon, Microsoft, Google, Meta, Oracle, etc) They are the biggest buyers of AI accelerators.

    Why Accelerators matter?
    Three big reasons:
    • AI models scale with compute: More compute → better models
    • General CPUs are too slow for neural networks: Accelerators can be 10–100× faster for some workloads.
    • Energy efficiency is critical: Power is now one of the biggest constraints in data centers.

    Types of Accelerators

    GPU (Graphics Processing Unit: Originally built for graphics but excellent at parallel math as thousands of small cores process many operations simultaneously e.g. Nvidia H100 / Blackwell or AMD MI300.
     
    ASIC (Application-Specific Integrated Circuit): A chip designed for one specific workload. Although offering lower power consumption and lower cost per task (at scale), they are less flexible than GPUs e.g. Google TPU (Tensor Processing Unit), Amazon Trainium / Inferentia or custom chips designed with Broadcom
    Other accelerator architectures (less common but important)
    • FPGA (Field-Programmable Gate Array)
    • Reconfigurable chips used in networking, edge AI, or prototyping.
    • NPUs (Neural Processing Units): Small accelerators in phones and laptops for local AI tasks.

    Accelerators Outlook

    • Some intelligence agencies such as Bloomberg projects the AI accelerator market could reach about $604 billion by 2033, growing at roughly 16% CAGR.
    • Other analysts differ in methodology but all show rapid expansion such as Mordor Intelligence at ~$440 billion by 2030 or Fortune BI ~$309 billion by 2034. The exact number varies because definitions differ (GPU-only vs. all accelerators vs. full AI infrastructure), but the direction is consistent: multi-hundred-billion-dollar market this decade.
    • Although GPU is leading with a 60% market share, ASICs are projected to grow at almost 30% CAGR within the 2026-2030 period.
    • Drivers behind this significant growth:
      • Datacentre AI CapEx.
      • Training and inference demand is exploding.
      • Frontier model training costs are rising sharply, with hardware a dominant cost driver.
      • The broader AI industry itself is projected to grow into the trillions by the early 2030s.

    GPUs vs ASICs: the structural shift
    • GPUs remain dominant: GPUs still account for the majority of accelerator revenue and are expected to represent around 80% of the market in some projections with Nvidia remains far ahead due to: CUDA software ecosystem, integrated networking and systems scale of production and customers.
    • But custom chips (ASICs, TPUs) are growing fast due to multiple drivers: Lower cost per inference, better energy efficiency and large cloud providers designing their own silicon e.g. Broadcom and partners are shipping millions of custom AI processors and scaling revenue rapidly.
    • Inference may overtake training: As discussed in a previous post, inference workloads could reach ~80% of compute demand within several years in some projections, which has several implications such as:
      • ASIC growth accelerates
      • Power efficiency becomes critical
      • Edge and on-premises deployments grow.
    • Most analysts agree on this structure:
      • GPUs dominate training.
      • ASICs increasingly dominate inference for large scale workflows for hyperscalers, yet GPUs will continue to be deployed for small scale workflows that require flexibility and lower upfront costs (e.g. small and mid firms). 
      • Even if inference dominates compute hours, frontier training drives technology leadership labs want the fastest hardware available. That fact should keep Nvidia strategically important even if inference volume surpasses training, according to consensus.
    The Players
    The market is oligopolistic and increasingly integrated (chips + networking + software + systems).
    In AI GPUs, the field is more concentrated than ASICs, and most revenue is captured by a small number of companies:
    • Nvidia: Could hold 70–75% of accelerator share by 2030 in some projections with clear leadership due to CUDA software ecosystem, strong developer adoption, full-stack systems (chips, networking, software, racks), and fast product cycles.
    • AMD: Growing but still behind Nvidia in software ecosystem and deployment speed.
    • Intel: Smaller share but still an important participant in accelerators.

    In AI ASICs, the competitive landscape is different from GPUs because many of the most important chips are designed in-house by hyperscalers and never sold broadly: Google (TPU family), AWS (Trainium for training and Inferentia for inference) or Microsoft (Maia AI accelerator). That said, there are semiconductor developers focused on ASICs designing and manufacturing for third parties such as:

    • Broadcom: Major designer of custom AI accelerators and networking silicon. Works with multiple hyperscalers on bespoke chips
    • Marvell: Custom compute and interconnect chips.  Strong position in data-center infrastructure.
    • Qualcomm: AI accelerators mainly for edge devices and PCs. Important in on-device inference rather than cloud training

    Feel free to share your thoughts



    ------------------------------
    Carlos Salas
    Portfolio Manager & Freelance Investment Research Consultant
    ------------------------------


  • 2.  RE: Semis Update - AI Accelerator Outlook

    Posted 4 days ago

    Good morning,

    Thanks for this overview. As you said, the software/ecosystem logic is probably the most important bottleneck at this point.

    I have both an NVIDIA GPU and an Intel TPU (New Ultra processors) on my computer, and even though I'd rather use the energy-efficient Intel NPU, it is hard to find software of libraries that are capable of exploiting that hardware. In Python for instance, for most ML tasks, most accept a CUDA pipeline but rarely AMD and almost never Intel NPUs.



    ------------------------------
    Guillaume (William) RUQUILLA
    MSc Fintech - MiM / Finance
    Audencia BS x Aston University
    ------------------------------



  • 3.  RE: Semis Update - AI Accelerator Outlook

    Posted 4 days ago

    Indeed, I like libraries like cumML as it allows me "to plug" my local GPU with a similar interface to scikit-learn, but I do agree sometimes it also can fall short depending on the needs of the researcher/developer.



    ------------------------------
    Carlos Salas
    Portfolio Manager & Freelance Investment Research Consultant
    ------------------------------



  • 4.  RE: Semis Update - AI Accelerator Outlook

    Posted 3 days ago

    Very good resource, thanks! I knew about Ollama for local LLMs processing but not about this one, it will definitely help.



    ------------------------------
    Guillaume (William) RUQUILLA
    MSc Fintech - MiM / Finance
    Audencia BS x Aston University
    ------------------------------