And getting better at interviews means adding structure. Time and time again, research has shown that structured interviews better predict job performance and reduce bias.

Yet almost no one does them. Why?

One barrier is the effort required for job analysis and interview question selection. Big companies are good at this and can afford to be because they recruit many people for similar roles. Smaller company interviewers are often frantically googling "interview questions for X" 10 minutes before an interview.

To level the playing field, I'm open-sourcing an AI to generate a complete 18-page interview playbook based on a job description and a company's values. This is a sample guide generated from a job description for an infrastructure software engineer on the Y Combinator website.

How can you use this?

• ChatGPT. Available on the GPT Store here: https://lnkd.in/gYKaAWaf

• Python Notebook. Open source and available on GitHub here: https://lnkd.in/gRrUPgbi

It is incredible to see the quality of the output. <$1 of API calls to produce something that would have taken hours. And, in my tests, it does a better job than most of us could do even if we had the time.

I would have loved this as a founder; I hope you find it useful. In return, I'd ask you for feedback - how could it be improved? Is this something you could use? I'm happy to burn some API calls and generate examples for your current job openings. Hit me up!

Also, kudos to Michael Hideo Smith, who introduced a form of structured interviews called Targeted Selection at TinyMCE and lifted our hiring game.

To elicit insights in these sorts of interviews, my cornerstone question is:

Take us back to high school and your interests then. What subjects did you take? What did you do after graduating? Take us on your journey from that point to sitting here today.

I call this the Life Journey Question.

The Life Journey Question allows the candidate to make the connection between their past and what led them to this role, highlighting their continuity in motivations and personality. People are generally great at telling their story and love the opportunity to share.

I’ve also found it a good ice breaker that helps calm candidates' nerves; everyone knows how to tell their story. It is a fantastic way to get to know candidates, understand their motivations and get a sense of them as a whole human. It is a way better way to learn about their experience than going back through their job history. Our lives move forward, not backward.

I hate repeating questions I know others on the hiring team have asked them, perhaps multiple times. I’m also often not the best person to evaluate specialized technical skills. By asking the Life Journey Question and paying keen attention to their answers, I hope to surprise them a little. I want to leave them the feeling that they would be valued and viewed in three dimensions if they were to join our company.

Clear communication skills are also essential for any role. A candidate who struggles to convey their personal story logically raises doubts about effectively sharing job-related ideas. Threading one's life journey into a coherent narrative demonstrates clear thinking. At a minimum, candidates should communicate a personal story that flows.

Finally, I’ve found this question makes it clear whether they are motivated by the opportunity. You can feel the misalignment of the job with their life journey. It just isn’t the next chapter in their story.

What are your favorite “fit” interview questions?

The Transformative Power of LLM

LLM-based apps have already demonstrated immense potential. Mario Rodriguez, VP of Product at GitHub, revealed in his talk that Copilot now has over 1 million users and generates over $100 million in annual recurring revenue. And the release of ChatGPT in November 2022 brought conversational interfaces into the mainstream.

However, there are inherent tradeoffs with these models to consider. Massive models like GPT-4 unlock advanced capabilities but are expensive and slow. Smaller domain-specific models are faster and cheaper to run but less flexible. Cloud-based models offer cutting-edge scale, while local models provide more control and privacy. There is no one-size-fits-all solution yet - the best approach depends on the context. Advanced applications will work with many different models.

Exciting LLM applications span code generation, task automation, and various forms of creative expression, like text, images, and music. Multimodal models that understand connections between vision, audio, and semantics open even more possibilities.

Real-World Deployment Challenges

While the demos at the conference were impressive, presenters repeatedly emphasized that building real-world systems using LLMs involves significant challenges like:

• Rigorously testing unpredictable model behaviors

• Implementing guardrails and verification for reliability

• Overcoming issues in data and prompt engineering

• Improving retrieval algorithms and metadata usage for relevance

• Establishing monitoring and benchmarks to refine components

Techniques like running evaluations (or "evals") using synthetic data, adding constraints, and leveraging human feedback loops are important to make these applications robust and trustworthy enough.

The Vibrant AI Community

Events like this play an incredibly valuable role in strengthening the nascent AI engineering community. I was struck by the brainpower in the room and how freely everyone shared their best ideas.

User feedback is also critical, as evidenced by the success of open ecosystems like AutoGPT and LangChain which have attracted huge community engagement. Feedback within LLM-based apps identifies edge cases and opportunities for improving prompts and fine-tuning models.

There is also a commitment across the field to pursue AI development ethically. This encompasses assessing risks, establishing governance principles, and educating practitioners on pitfalls like bias.

The Future

Some promising trends I saw at the conference included:

• Emerging multimodal applications combining language, vision, and audio. GPT-4V is seriously impressive; you should play with it on ChatGPT today.

• Innovative contextual reasoning and user-adaptive interfaces

• Agent training via reinforcement learning in simulated environments

• Synthetic dataset creation by distilling knowledge from large models

Going forward, key goals for AI-enhanced engineering include moving to more abstract, outcomes-based programming paradigms. Interpretability of model behaviors will grow in importance. There will also be a focus on mainstreaming access to AI and assessing potential risks.

Next Steps

The AI Engineer Summit will be back in San Francisco next year in a larger venue, and I highly recommend it. In the meantime, you can watch the replays of Day 1 and Day 2.

Subscribe now

Moving Beyond PLG to PLS

The concept of product-led growth (PLG) took the SaaS world by storm a few years ago. The idea is irresistible - build a product so great that it sells itself through word-of-mouth, virality, and organic adoption. A shiny new object that renders traditional sales teams obsolete.

It's a compelling vision but also an incomplete one. While PLG has proven a powerful model for efficient acquisition, conversion, and launch, few companies have scaled into sustainable, high-growth businesses.

Congratulations if you’ve gotten your SaaS off the ground on PLG alone; it is not easy by any means. But to tap into the majority of revenue available in your market, you will likely need to layer in a complementary product-led sales (PLS) motion. One that combines the organic, bottom-up power of PLG with the benefits of sales assistance.

Done right, PLS reduces friction, boosts conversions, expands deal sizes, and enables efficient scaling into enterprise accounts. It brings together the best of both worlds - product-led virality and higher-touch sales relationships.

In this post, I'll explore the benefits of PLS, common myths, identifying PLS opportunities, implementation advice, and why this hybrid approach unlocks the most growth.

The Benefits of Adopting Product-Led Sales

What makes the PLS model so effective for growth? Throughout my career, I've consistently found that 70% of revenue comes from the top 10% of our customers. These large enterprise accounts typically require higher-touch sales assistance when making purchasing decisions. A "no sales" mantra misses out on tapping into this vast revenue pool. PLS provides customized guidance and account management that unlocks enterprise spend you would not otherwise get.

By having sales reps build closer customer relationships, PLS also improves retention and maximizes the expansion revenue driven by product usage and cross-selling. Our sales-managed accounts regularly achieved over 100% net dollar retention, while self-service customers churned as much as 5% a month. This increased lifetime value is challenging to match with PLG alone.

Large enterprise accounts expect white-glove service from sales reps when they make purchasing decisions. PLS provides enterprises with the customized assistance they want and stops them from going elsewhere.

Common Myths About Product-Led Sales

Given the novelty of PLS, there are some common myths worth addressing. 

Many assume sales reps need to be located near customers. With tools like Zoom, I’ve seen that location matters less than cultural alignment. Sales can happen efficiently from anywhere. Indeed, our sister company, CKSource, has built a great business selling worldwide with its sales team entirely based in Poland.

You might even think PLG and sales are incompatible. On the contrary, they’re complementary. PLG seeds opportunities for sales to expand upon. Sales then fuel further virality and referrals.

Some believe integrating sales will erode company culture. But with careful hiring focused on cultural fit, sales and PLG teams can mutually thrive. I found that a stellar sales team made us even more customer-centric.

The biggest myth is that PLG precludes ever needing sales. However, even PLG pioneers like Atlassian, Slack, Zoom and Dropbox now have sales teams. They realized sales unlock additional growth at scale.

Identifying Product-Led Sales Opportunities

When should your sales team reach out to product-qualified leads? Many of the best prospects will self-identify through your “Contact Us” form, but there are also some key signals to watch for:

Look for power users who are deeply engaged with the product. They can become internal champions who advocate for purchasing and expansion. If users ask complex technical questions online or via support, they likely need the customized guidance a sales rep can provide.

When users submit feature requests, it may indicate they've outgrown current self-serve plans and need capabilities from higher tiers. I often refer product leaders to EnterpriseReady for inspiration for enterprise-level features.

When users exceed thresholds like the number of projects or collaborators in self-serve plans, it means they're primed for upselling. If users seek to integrate your product with other tools via APIs, they may be ready for customized enterprise plans.

Advice for Implementing Product-Led Sales

When hiring sales reps, prioritize finding people who align culturally and embrace a product-led mindset over those who just want to "hunt." Invest in resources and training to educate users on the evolving value as your product expands.

Use pricing tiers like "Free, Good, Better, Best" with a "Contact Sales" CTA on the Best tier to encourage upgrades. Have leaders personally engage promising inbound leads to build knowledge and ensure they pay attention to what is being said via Gong.io (or whatever you use.)

When transitioning sales reps from traditional techniques to a product-led sales approach, keep these training tips in mind:

• Prospects are often highly informed about the product already. Avoid overselling or relying on scripts. 

• Focus on efficiently solving the specific problems or use cases that brought prospects in. Don't distract them. You may not need a demo, and our reps haven’t used a PowerPoint presentation for years.

• Leverage tools like Gong.io and Outreach to refine approaches for optimizing inbound leads.

• Prioritize cultural alignment and a customer-centric mindset over being hard-charging—structure compensation to reward expansion within accounts, not just net new logos.

• Highlight how sales can tap into revenue potential from enterprise accounts missed by PLG.

• Encourage collaboration with product and marketing to educate users on new value continually.

I found it always helpful to share examples of other companies succeeding with a hybrid PLG + sales approach. Our exec team often discussed MongoDB (this video in particular.)

The Hybrid Model is the New Normal

In today's world of self-service software, product-led growth is an alluring vision of viral acquisition and frictionless expansion. But very few companies have managed to sustain growth on PLG alone.

To tap into the majority of revenue potential in your market, you'll likely need to layer in a complementary product-led sales approach. One that reduces friction for larger buyers, expands deal sizes, and enables expansion.

PLS brings together the best of both worlds - viral product-led motion with higher-touch sales relationships. Done right, it unlocks growth from both expanding existing accounts and penetrating untapped enterprise opportunities.

Further reading:

• Guide to Product-Led Sales from OpenView

• McKinsey's From product-led growth to product-led sales: Beyond the PLG hype

• Introduction to Product Led Sales by Arthur Ventures

• Video: Consumerizing MongoDB to Grow to $100M+ by Sahir Azam

Not so fast, says Meta’s Chief AI Scientist, Yann LeCun. When I get excited about new technology, it is valuable to hear the counterarguments, and, as a winner of the Turing Award - essentially the Nobel Prize of computing - Yann knows what he's talking about. Under his guidance, Meta is also a leading developer of large language models, so it is worth paying attention when he criticizes them.

In a keynote address at MIT last week, LeCun discussed the fundamental limitations of current approaches and his vision for moving beyond what he calls “narrow AI” to achieve more robust, flexible, and human-like intelligence. Given the many open challenges, he says it may take "years to decades" to reach human-level AI. 

In this article, I'll explore why LeCun feels current methods fall short and what different architectures and training techniques may get us to the next level. He said that LLMs “suck” and are “doomed.” Are they?

The Limits of Narrow AI

LeCun used autoregressive language models like ChatGPT as a prime example of impressive, narrow AI. Autoregressive means these models generate text or data sequentially, with each new token based only on the previous ones. As a result, they cannot reason about the overall meaning of what they're generating. LeCun compared them to Kahneman's System 1 thinking - quick, subconscious reactions rather than conscious reasoning.

While they seem amazingly fluent and knowledgeable at first glance, language models like ChatGPT suffer from significant shortcomings:

• They hallucinate plausible-sounding but false or nonsensical answers

• Without broader reasoning, they lack common sense and knowledge of how the world works

• They can't incorporate recent information outside their training data

• Making them 100% safe is impossible - they can always be "jailbroken."

As LeCun said, "Their intelligence is very limited and really nothing like human intelligence.”

What's more, LeCun explained that these models have no connection to physical reality. They acquire knowledge purely through language rather than observing and interacting with the world as we do. Consider how babies learn object permanence, gravity, and basic physics through observation months before they can even speak. Animals likewise navigate the world adaptively without using language. Autoregressive models cannot access this intuitive "background knowledge" we take for granted.

Moving Beyond Narrow AI

Given the fundamental limits of today's AI, LeCun argues that new approaches are needed to create more capable, general architectures. He believes the key is to move beyond narrow pattern recognition and text generation tasks towards AI that can reason, plan, and understand the physics of the world.

Several of LeCun's proposals resonated with my intuition for how human and animal minds work:

• Cognitive architectures incorporate different modules to perceive the world, model dynamics, assess costs, and plan actions like teams of specialists cooperating flexibly.

• Objective-driven systems that optimize hardwired and learned goals and constraints. This prevents uncontrolled or unsafe behavior.

• Hierarchical planning to break down goals and develop solutions at multiple levels of abstraction.

• Multimodal self-supervised learning that models physics and relationships by observing raw sensory data like images, video, and sound - not just text.

The last point echoes how my sons have learned what different animals look like from picture books long before they could read their names or facts about them. The visual experience encoded the concepts in their brains on a deeper level than language alone. LeCun stresses that capturing this implicit knowledge is vital for more human-like learning in AI systems.

New Training Methods

Today's large neural networks are famously data-hungry. For example, Llama-2 trained on massive amounts of online text totaling over two trillion tokens! LeCun says these models cannot learn from experience and incorporate new knowledge over time as humans do, and their knowledge is fixed during training. I wonder if he gives enough credit to prompt engineering techniques such as retrieval-augmented generation (RAG) and their ability to work around limitations. Still, he has a point about the language models themselves. 

LeCun proposes training advanced AI architectures using very different methods that don't require massive datasets:

• Self-supervised learning techniques can extract representations from unlabeled data like images, video, and audio. This approach was inspired by how animals learn about the world through observation long before receiving explicit labels or reinforcement.

• Energy-based models aim to learn relationships directly from underlying dependencies in the data rather than relying on huge datasets to cover all possibilities. LeCun says this is closer to how human reasoning works.

• Joint embeddings, where representations are predicted in abstract vector spaces rather than generating pixels or tokens directly, avoid many pitfalls of mainstream generative models.

• Regularization techniques like VICReg and DINO constrain models during training. They help prevent the learned representations from becoming too correlated, collapsed, or meaningless - which is especially useful in self-supervised learning where the model trains on unlabeled data without guiding feedback from labels.

By incorporating these techniques, LeCun and his team at Meta aim to create AI that learns broadly about relationships in different domains in a more efficient, human-like manner rather than brute-force pattern recognition from vast data.

The Path Ahead

The keynote was technically dense in parts. Nevertheless, the main points that came through for me are:

• Today's narrow AI still has fundamental limitations compared to human intelligence.

• Truly capable, general artificial intelligence will require rethinking system architectures and training.

• Objective-driven, hierarchical, multimodal learning systems hold promise for achieving more human-like flexible intelligence.

• New methods like self-supervised learning and energy-based models can reduce dependence on massive datasets.

LeCun's perspective is a refreshing take on the hype. AI has come a long way, but the path to human-level artificial intelligence is still long.

It is worth finishing that LeCun is no AI doomer. He believes that risks associated with advanced AI can be mitigated through responsible R&D practices. He is optimistic that future human-level AI will greatly benefit humanity. Large language models may be doomed, but we are not.

Note: Anthropic's Claude-2 and Glasp's YouTube Summary browser extension helped write the first draft of this post. Article image generated by Midjourney

A few weeks ago, I stumbled on a great video on AI user experience (UX) design from the Microsoft Build conference. Rachel Shepard, Director of Design for AI Platform, and Kurtis Beavers, Fluent AI Design Director, discussed designing for Copilot - Microsoft’s name for embedded AI assistants surfacing throughout their products. I highly recommend that you check out the whole video.

As design leaders for Microsoft AI products, they’ve seen firsthand how to create copilots, or AI assistants, with quality UX. In this post, I’ll share some of their insider tips on crafting a copilot, from managing expectations to continuous improvement. Deceptively simple, there is so much to be thought through that I believe that AI UX will blossom into a whole specialty unto itself.

Defining Copilot

To start, what exactly is a copilot? Essentially, an AI agent collaborates with you on tasks through natural conversation. Under the hood, copilots leverage large language models like GPT-4 to generate responses based on their vast training data and augment it with your application's data. But to users, it just feels like chatting with a helpful assistant. 

Compared to traditional apps, copilot interactions are more unpredictable, like a choose-your-own-adventure story. Users drive personalized experiences based on their questions and requests. Designers have to embrace this ambiguity rather than try to control rigid paths. In the video, Rachel calls for a probabilistic mindset attuned to likely outcomes, not prescripted certainties.

Laying the Foundation 

The Microsoft Fluent 2 Design System provides the foundation when constructing a copilot, at least for Microsoft products. They have expanded it beyond the typical focus on visual components to include education, conversation norms, trust building, and, crucially, prompt design. Their core AI UX principles include:

• Establish appropriate trust in the system

• Help people spot inaccurate or potentially harmful content

• Help people form better inputs

• Help people understand & use outputs

Framing the experience properly is critical so users understand the technology’s current stage. We are coaching users on collaborating effectively rather than setting sky-high expectations. The proper framing shapes how people perceive the system’s strengths and limitations.

Copilot Altitude

Copilots can engage with users across different modalities, or "altitudes," as they call them. Full-screen chat interfaces for free-form dialogue are on one end of the spectrum ("above"). This works well for open-ended queries or back-and-forth brainstorming.

In other cases, a side-panel copilot (“beside”) might monitor the core app experience and interactively assist users. Copilots can also be embedded "inside" specific UI elements for focused micro-tasks to provide just-in-time guidance.

Determining the appropriate modality depends on the use case and context. Sidebars suit complex workflows by advising on app functionality. Embedded copilots help with precise steps. Chat excels for discovery and exploration. Mixing modalities allows each interaction to feel fluid and natural.

The Deceptive Simplicity of Copilot UI  

Copilot UIs look straightforward on the surface. Behind the minimalist interfaces, though, designers must craft details thoughtfully—elements like previews, citations, and notifications. 

The teams at Microsoft spend lots of time working through nuanced functionality around latency, accuracy, grounding in source materials, and more. How might they balance a snappy response time while delivering an in-depth quality result? What sources or data should responses link to so users can verify credibility? How do they indicate when a copilot hallucinates an answer rather than pulling from reliable information?

Every design choice aims to build users’ trust at the appropriate level. While copilots can generate helpful information, they sometimes fabricate or hallucinate content that requires correction. UI tweaks help flag these issues, allowing people to steer toward truthful insights.

The Art of Prompt Engineering  

Prompt engineering takes experimentation and patience. You could compare it to perfecting a recipe. The ingredients and instructions must precisely guide the AI to serve the knowledge you’re hungry for. Going too broad or too vague produces mushy meaningless results. Overly narrow or complex prompts also risk failure. 

Start by writing prompts that clearly describe the problem to be solved or the question to be answered. Build in guardrails to bound the scope if needed. Provide examples that illustrate the desired tone and depth. Work iteratively, tweaking prompts based on the outputs generated until you consistently get delicious results. 

Lately, I have been successfully expanding my often simplistic prompts into much more detailed instructions for the language model. A well-crafted prompt makes a massive difference to the output from an LLM, and there are many techniques you can use to improve them.

The People Powering Copilots

Copilots demand increased collaboration between groups that previously didn’t intersect much, like design, research, engineering, content strategy, and trust and safety teams. 

This rallying of different experts unlocks creativity and checks potential pitfalls. Designers ground lofty AI ambitions in realistic user needs. Engineers advise on technical constraints that enable seamless experiences. Researchers supply data-driven insights and identify pain points. Prompt engineers finesse conversation flow. Ethicists monitor for potential harm. The combined superpowers drive breakthroughs.

Developing AI responsibly requires this collaborative spirit. Copilots augment human capabilities rather than replace them. So designing copilots together makes the output more insightful than any individual could produce alone.

Launching and Iterating 

Shipping copilots requires a build-measure-learn approach. Since you can’t fully predict how users will converse, regularly releasing minimum viable products lets you gather feedback and continuously refine. 

Plan ahead for how improvements will integrate back into the copilot post-launch. Monitor both qualitative signals like user delight as well as usage metrics. Be ready to update content and prompts based on common queries. Expand capabilities over time as the models advance. 

Occasionally, you may decide certain features are no longer needed if the copilot proves adept at handling associated tasks. It takes humility to recognize when technology evolves past previous solutions. But minimizing duplication avoids clutter and complexity.

The Future of AI Assistance   

It’s an exciting time to think about how AI assistants integrate into our lives. We will continue dreaming up ways to combine human strengths with intelligent machines. AI can help us achieve our goals while avoiding drudgery. Approached thoughtfully, it will enhance how we learn new skills, complete workflows, run our businesses, and more. 

I can't wait to see how "AI UX" designers build out the patterns and frameworks we will need to be successful. We are still in the Geocities era of AI.

Pretend you are a computer game that uses text to describe a scene. 
The player can then interact with the scene using natural language. 
The initial starting point for the game is the entrance to 
Australia Zoo.

What followed was a realistic interaction about a real place. It knew the shows, attractions, and, most surprisingly, the geography. Crikey! I haven’t seen much coverage of generative AI in geospatial applications, but the models are wrestling with some emergent location-based understanding.

Fortunately, some good folks from academia had the same thought, and on May 30, they published the GPT4GEO paper. The collaboration between researchers at universities in the UK, Germany and China explores the factual geographic knowledge of GPT-4 and its application-centric geographic reasoning. They conclude, “We find GPT-4 to have a remarkable understanding of the world.”

The factual or descriptive knowledge of GPT-4 is fascinating, but I don’t think it matters too much. Retrieval augmented generation (RAG) could provide accurate geospatial data from Google Maps and other sources. I was surprised that there isn’t yet a plugin on ChatGPT to do this; it won’t be long before there is.

The application-centric geographic reasoning, such as route planning, navigation, itinerary planning, and supply chain analysis, gets interesting. The paper showed that the model can develop “creative, plausible travel routes” and “shows strong capabilities of direction-based navigation.”

Fire Safety GPT

I’ve been advising Locatrix, an indoor mapping software company. Locatrix focuses on fire safety, and I was curious whether GPT-4 could generate an evacuation plan based on an office floor plan or recommend where to place fire extinguishers. The short answer is yes, yes, it can. 

I asked GPT-4 to “Create the floor plan and detailed layout of a 5,000 square foot floor of an office building. Return it as JSON.” It did a solid job describing the area, including a reception, four meeting rooms, six private offices and an open office space.

I then followed up with “Create an evacuation plan for this office”, which it proceeded to do. At a high level, it looks pretty good and claims, “Each exit point has a route from every section of the office floor (Reception, OpenOfficeSpace, PrivateOffices, MeetingRooms). The path represents the sequence of positions to be traversed from each section to reach the exit point.”

To push the limit, I asked, “Suggest locations for fire extinguisher locations to meet the fire code in San Francisco.” Sure enough, it returned suggested plausible locations and details on how it complied with the local code. “As per the specific codes applicable to San Francisco, each extinguisher should be: within 75 feet of all portions of the premise, within 30 feet of any cooking equipment, and regularly inspected and maintained.“

You can read the full transcript of my interaction with ChatGPT and try it out for yourself. It could be better. For example, I couldn’t get it to produce any decent visualizations. Nevertheless, you can see the future from here.

Coding and data analysis are tasks already well suited to the GPT architecture. Geospatial features can all be represented as text too, and I suspect that a few years from now, every large building or campus will have a copilot to help with space planning, safety, and maintenance. 

“Robot, we need to evacuate. Help everyone get out now!” is not far off. Every building will be an AI building.

[1] GPT4GEO: How a Language Model Sees the World's Geography by Jonathan Roberts, Timo Lüddecke, Sowmen Das, Kai Han, Samuel Albanie. Submitted to Arxiv on May 30, 2023.

Recently, I've had a change of heart. Open source is no underdog. In fact, it’s stepping up, creating models that match and potentially surpass the capabilities of the best foundational models. How did we get here?

Trust and transparency will motivate larger end-user organisations to work with open source models. Open source solutions are often easier to self-host, and many organisations value keeping data behind a firewall or within their country’s borders. The models and training methodology will also be open and accessible to anyone and allow for scrutiny. For industries dealing with sensitive information, this control over data is paramount. This preference among large users will give open source a business model. Vendors can and will commercialise open source projects with proprietary add-ons (“open core”) and support. At Tiny, we saw the power of this approach in action and signed up over 2,000 paying customers.

As a result of this now tried-and-true business model, commercial open source AI vendors will almost certainly generate tens of millions in revenue and attract venture capital to support their growth. That’s enough for many great engineers, but surely more is needed for the compute time to compete with hyperscalers? The enormous costs associated with competing with tech giants still seemed like a deal breaker, but two factors are pushing back.

First, open source models can tap into a massive community of developers, researchers, and users. The new algorithms and approaches they invent, such as low-rank adaptation (LoRA), can significantly reduce training costs. Stanford’s Alpaca model, announced in March, can be trained for less than $600 and behaves “qualitatively similarly to OpenAI’s text-davinci-003 while being surprisingly small and easy/cheap to reproduce.” Another promising contender, Guanaco, was announced today, and you can play with it here.

Second, big tech companies trailing behind Microsoft and Google, such as Meta, Amazon, Intel, IBM or Alibaba, see the strategic value in supporting open source AI. A prime example is Meta's LLaMa model, which was transformed by academia into an even more powerful solution within weeks. This resulted in the infamous “we don't have a moat, and neither does OpenAI” memo from a Google employee. 

So, the conclusion appears clear. Open source AI isn't just surviving, it's thriving. But is this a good thing? The dangers of AI are real and growing. Powerful models in the wrong hands can aid fraud and disinformation, as demonstrated in this 60 Minutes clip.

It will also power the war machines of the future. Russia’s invasion of Ukraine and the growing tensions across the Taiwan Strait have set us on a new geopolitical path. Powerful open source models enable enemies as much as allies. Should we fail in AI alignment, we could inadvertently give birth to autonomous weapons with their own agendas and no regard for human life. It is easy to argue that open source will make alignment harder, not easier.

Open source has contributed to tremendous progress over the past twenty years. At Tiny, we were proud of our open source software's positive impact – from helping fight California wildfires to assisting social workers in Sweden. These stories often infused us with a sense of purpose.

My own career also owes a lot to open source software, and I want to engage more in open source AI. Yet, I grapple with this question: Are we promoting open source AI to fuel a potential catastrophe? Or is open source AI, despite its risks, a force for good in the world, preferable to a future dominated by a handful of megacorps? I’m torn, and I'd love to hear your thoughts.

Around 20 years ago, I swapped out my coder's beanie for a founder's cap. I launched a company and left behind hands-on software development. It was a wild journey where I learned so much, but my coding skills? They started gathering dust. As I step back into the developer's arena, it feels like I'm the new kid on the tech block.

Recently, I sold my company and felt this urge to dive back into product building. I missed the thrill of creating something new from scratch. Plus, AI's current wave has me utterly fascinated. Having had a few encounters with AI throughout my career, this time, it feels different. The shift reminds me of witnessing the birth of the internet. It's kindled tech enthusiasm in me all over again.

How is my coding comeback going? It's been really fun, all thanks to AI. Using tools like ChatGPT, GitHub CoPilot, and Replit’s Ghostwriter feels like having a magic wand. It helps me dust off my knowledge of server architecture, Unix commands, JavaScript, and my 90s-era computer science degree, turning it into something practical. Now I can ask questions, fix errors, and even enhance my code, all with a little help from my AI friends.

Some folks are worried that AI will replace programmers. I find that scenario unlikely, but I do see a massive productivity boost on the horizon. AI enables engineers to stay sharp longer and shift between technologies much more quickly.

Without AI, plunging back into product creation could've felt like navigating a dense jungle without a guide. Now, it's like taking a guided tour with a wise companion.

--

And, for some fun, I've included an adaptation of Dylan Thomas's famous poem written, of course, by ChatGPT.

Do not go gentle into that code night,
Software crafters nearing twilight's sight,
Burn and debug at the end of day;
Rage, rage against the obsolescence of the byte.

Though wise men at their terminals concede time's flight,
With AI assistance, their craft ignites,
They do not go gentle into that code night.

Good coders, the last wave by, sensing the bright
Their frail code might have danced with AI's light,
Rage, rage against the obsolescence of the byte.

Wild coders who caught and sang the sun in flight,
With machine learning, regained their height,
Do not go gentle into that code night.

Grave coders, near dusk, who see with blinding sight
Old algorithms could blaze like meteors with AI's might,
Rage, rage against the obsolescence of the byte.

And you, my coder, there on the sad height,
Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that code night.
Rage, rage with the AI by your side in the fight.

Until recently, I had thought “fine-tuning” a large language model (LLM) was necessary. I understand fine-tuning at a high level, and it is complicated to implement and keep up to date. A new category of “LLM Ops” tools is emerging to make this easier, but it still feels like a heavy lift.

Since handing over the reins at Tiny, I’ve dusted off my rusty developer skills and tried to get up-to-speed on AI. I want to share a recent “aha” on using prompt engineering and context stuffing of out-of-the-box LLMs such as ChatGPT to achieve extraordinary integration with your data. I believe this technique is also called “retrieval-augmented generation”, or RAG. You have to love tech industry acronyms!

When you hear “prompt engineering”, you probably think of the one-or-two sentences you type into ChatGPT to get a poem about cats in the style of Eminem. Indeed, there is helpful guidance to an LLM that you can give in just a few words, but you can give them a lot more context, significantly more! Anthropic recently announced a 100,000 token model, and the head of OpenAI, Sam Altman, has floated the idea of a 1 million token model in the future. (For the uninitiated, a token can be considered a word.)

By putting your software between the user and the LLM, you can retrieve data and assemble a big prompt. Or even a colossal prompt. While this might seem obvious, the penny didn’t drop for me until I listened to this video from Microsoft on how to use ChatGPT with your enterprise data. In their related GitHub repo, they have a helpful diagram that I’ve adapted here (with added inspiration from AWS):

The example used is an internal HR application that lets users ask questions such as “Do my benefits cover this expense?” The app server searches an internal knowledge system to find PDFs that describe the benefits. It then uses the snippets of relevant knowledge from these PDFs and a specially worded prompt to get the large language model to produce a custom answer specific to the organisation.

The beauty of this approach is that your app can respect user permissions. You integrate only the data your user has permission to see into the prompt you send to the language model.

Once you have your proprietary data, you can build the prompt with additional system instructions. In Microsoft’s example, they lock down the AI with the following instructions added to the prompt: 

Answer questions as shown in the following examples, by 
splitting the question into individual search or lookup actions 
to find facts until you can answer the question. Observations 
are prefixed by their source name in square brackets, source 
names MUST be included with the actions in the answers. All 
questions must be answered from the results from search or 
look up actions, only facts resulting from those can be used 
in an answer. Answer questions as truthfully as possible, and 
ONLY answer the questions using the information from 
observations, do not speculate or your own knowledge.

With these instructions, the AI will only use facts you have provided and not hallucinate new information for your users. You can see how Microsoft implemented this idea with Bing Chat.

Perhaps this is old news for you but, for me, hearing about this approach was an “aha” that existing data that be used to build a differentiated generative AI experience without training or fine-tuning or a custom model. OpenAI is yet to offer a way to fine-tune GPT-4, so they see the value in how much customisation can be achieved through context stuffing and similar techniques.

If this is still mysterious and my explanations confuse you, I recommend the “Can ChatGPT work with your enterprise data?” YouTube mentioned above.