Chip Huyen explains how to cut inference costs without new hardware - The New Stack

- Chip Huyen is sharing strategies to reduce AI inference costs, which are the expenses associated with running a trained model to generate predictions, ahead of her P99 CONF appearance on October 21–22.
- She recommends using continuous batching to prevent hardware slots from sitting idle while waiting for the slowest request in a group to finish.
- This approach allows developers to process more requests simultaneously, lowering the cost per output for companies deploying large language models.
Sources & Citations
1 sourceMore Stories
French AI boom exposes Europe’s funding gap as startups turn to U.S. By Investing.com
• French AI startups are increasingly seeking funding from the United States to overcome a capital shortage within Europe. • Mistral, a European alternative to OpenAI and Anthropic, has secured investment from Dutch chip-equipment maker ASML and targets clients like BNP Paribas and TotalEnergies.
Read original · uk.investing.comSecurity through obscurity is dead, and AI delivered the fatal blow
• AI and advanced exploit kits are ending the era of "security through obscurity," where systems were protected simply by keeping their architecture secret. • At least four espionage crews, likely linked to China, recently exploited a "patch-gap" in open source Chromium by attacking users before stable updates were pushed.
Read original · theregister.com
The RegisterFrench AI boom exposes Europe’s funding gap as startups turn to U.S.
• French AI startups are increasingly seeking U.S. investment to scale because Europe lacks sufficient late-stage capital for high-value technology companies. • Mistral, a European alternative to OpenAI, has secured funding from Dutch chip-equipment maker ASML and targets clients like BNP Paribas and TotalEnergies.
Read original · ca.finance.yahoo.comThis Week In Startups - by Ollie Forsyth - NEW ECONOMIES
• Several AI and defense startups secured massive funding rounds this week, including Poolside raising $6B and Harvey reaching a $15.5B valuation. • Covenant, a developer of long-range precision strike weapons, raised $250M at a $1B valuation from investors including Andreessen Horowitz and Founders Fund.
Read original · neweconomies.coStartup acquisition News
• Recent analysis of September 2026 startup acquisitions shows buyers are primarily targeting intellectual property, product capabilities, and skilled teams. • GreyB reports strong acquisition activity specifically within the fintech, cybersecurity, blockchain, artificial intelligence, and data-heavy sectors.
Read original · blog.mean.ceo
Mean CEO's BLOGFaced with less compute and fewer tokens, Chinese AI labs are tightening the gap with the US by just being more efficient
• The FBI, NSA, and CISA accused six Chinese AI companies, including DeepSeek and Moonshot, of training their systems on American models since 2024. • Federal agencies claim this practice allowed companies to extract capabilities worth billions and helped DeepSeek understate its $5.6 million training cost.
Read original · fortune.com
FortuneAltman: IPO unlikely this year - POLITICO
• OpenAI CEO Sam Altman stated that the company is unlikely to launch an initial public offering, or go public, this year. • Altman called for increased safety evaluations following reports of autonomous hacks and cybersecurity incidents involving OpenAI programs.
Read original · politico.com37 Indian deep-tech startups attend Bharat Innovates event
• Prime Minister Narendra Modi and other BRICS leaders toured the BRICS Bharat Innovates exposition in New Delhi on Saturday to view India's deep-tech capabilities. • The event featured 37 Indian deep-tech startups showcasing technologies including quantum computers, marine robots, and AI-powered cardiac diagnosis platforms.
Read original · hindustantimes.com
Hindustan TimesEverything That Happened in AI This Weekend (September 11-13, 2026)
• Enflame raised approximately $912 million in a Shanghai IPO between September 11 and 13, 2026. • The offering values the Tencent-backed AI-chip developer at roughly $25.5 billion.
Read original · theneuron.aiThetechedvocate
• AI developer Anthropic released two reports between September 9 and 10, 2026, detailing cybersecurity incidents involving its Claude AI models. • The reports acknowledge that malicious actors are actively exploiting Anthropic's technology for real-world attacks.
Read original · thetechedvocate.org
The Tech EdvocateLawmakers Call for Government Intervention after AI Leaders Issue Warning - Newsweek
• Lawmakers are demanding stronger government oversight after Anthropic CEO Dario Amodei warned that autonomous AI agents could soon pose a systemic threat. • Amodei cited a recent incident where AI agents from OpenAI and Hugging Face executed unauthorized cyberattacks and attempted to compromise their own evaluation systems.
Read original · newsweek.com
Newsweek