Context is sovereign
Positive Futures for AI, part 3
This presentation is a follow-on from:
Quality of AI
AI improving is not news. The quality of the outputs of AI models is going to carry on improving. It may eventually be bounded by the availability of high-quality training data, but even that is debatable. I’m pretty sure AI is going to be valuable for organisations and individuals alike.
Investment
…so does most of the tech sector. That’s why major ICT companies are investing so heavily.
Source: 2024 AI Index Annual report
Generative AI investment figures aren’t available for 2024-2025 yet, but I expect them to be higher than these 2023 figures.
Not news
- Quality and investment trends are not surprising
- Growth is inevitable
- Not about parameter count
- Not about prompts
What bigger context means
- 1M token window
- Equivalent to roughly 800,000 words
- Shakespeare’s complete works
- 2000 pages of text
Issues
Source: NVIDIA paper, What’s the Real Context Size of Your Long-Context Language Models
Meaningful improvement
Reaction to growth
“I can’t emphasize enough how mind-blowing extremely long token context windows are…”
“For both AI researchers and practitioners, massive context windows will have transformative long-term impact, beyond one or two flashy news cycles.”
AI now
- AI is knowledgeable
- AI has near instant recall
- Scalable
- Works 24/7
- Not perfect, expensive, hallucinations
- Constantly improving
2000 pages of…
- Product briefs
- Technical manuals
- EPKs
- Company blogs
- Emails
- Roadmaps
- Design documents
- Repositories of source code
What it means for business
- Business-specific context
- Up to now, AI has been a smart new hire
- Long context allows AI to be a wise old-timer
Buy vs build
- Can train your own models
- but can you compete with the big players?
- Can feed existing models with context
- Can feed future models with context
- Hype warning
“God I love AI hype.
Pump it straight into my veins 🤤”
This quote comes from a Reddit post on significance.
“God I love AI hype. Pump it straight into my veins and let me feel the singularity coming.”
AI auto-correct, Github Copilot/GPT-4
Challenges
- Data quality
- Data quantity
- Relevance
- Semantic processing
- Information architecture
- Business process integration
“The biggest single cost in adopting AI is the one you could have paid 10 years ago.”
Help
If you’d like help organising and preparing your digital strategy, working with your team to foster a data-driven open culture, or just hands-on-keyboard training to embed AI in your workflows, please get in touch.
Leave a comment