LLM TRAINING
→ Content strategy for an LLM that facilitates fitness guidance

💬 Introduction

At the start of 2024, Freeletics set its north star: a fitness trainer in your pocket. The challenge for the product team was clear—how could we leverage our app and emerging technologies to bring this vision to life? Enter LLMs (Large Language Model) and AI (Artificial Intelligence), powerful tools that could help users get reliable answers to their fitness and product questions, emulating the experience of having a personal coach available 24/7.

As the content designer—and the product content expert—I was entrusted with leading the content strategy to train Coach+, Freeletics' Generative AI assistant. With a short timeline before launch and vast amounts of data in multiple formats and languages, we needed a content strategy that was both efficient and adaptable. I took the lead in structuring and refining the model’s responses so they were not only technically accurate but also helpful and aligned with our brand voice.

💡Concept
A fitness trainer in your pocket

🎯 Goal
Provide product and fitness advice to users

⚠️ Considerations
Short timeline before launch
Large amounts of data in different formats and languages

✅ Solution
A content strategy that is flexible, fast and easy to implement

🔎 Scope

The first step was understanding how LLMs work and defining the essential topics for the initial release. This required identifying what information the model already had and what was missing. I worked closely with developers, engineers, data scientists and the product team to align technical feasibility with content needs.

The product team had strategically placed Coach+ within the app where it could have the highest impact. With these use cases mind, I mapped out the content needs to establish priorities, grouping the information in two main categories: general fitness information and product-specific content.

Next, I broke down these broad categories into manageable deliverables to maintain focus and meet required deadlines. At the same time, I developed a general quality framework to set clear response standards and make sure users would receive the best content possible.

To support scalability, I established a clear information hierarchy and created a product concept map that organized content in a way that made it easy to update with new information and integrate future features seamlessly.

🗓️ Strategic Planning
Defined key topics and content gaps
Mapped content to specific use cases
Prioritized deliverables for implementation

📑 Content
Grouped content into 2 main categories: general fitness and brand specific information
Outline QA framework to set response standards

🤝 Cross-Team Collaboration
Worked with developers, QA engineers, designers and data scientists to define technical needs and feasibility

📝 Training Process

Training an LLM involves two key components: the preprompt—a set of instructions that define how the assistant should behave—and the custom corpus—a curated repository of knowledge it draws from to provide answers.

For the preprompt, we drafted a directive that would provide information in a positive and informative tone. For the custom corpus, I started by defining the scope of fitness knowledge needed. The first step was collecting common fitness-related questions users ask online to identify trends and create a list of high-priority topics.

With these insights, I curated relevant articles from the Freeletics blog, selecting content that aligned with our goals and feeding it into the LLM. For the most frequently asked topics, I crafted dedicated articles that provided clear, actionable advice tailored to our brand voice.

On the product side, I conducted a comprehensive audit of the app, breaking down its features and sections to identify potential areas where users might need guidance. I then categorized and prioritized these topics based on their importance to new users. This structured approach allowed me to set clear milestones for the initial release while allocating enough time for quality testing and refinement.

🧠 Training Components
Preprompt to define assistant behavior
Custom corpus for fitness and product knowledge

🔍 Content Research
Collected common fitness questions from online sources
Identified key user concerns across multiple platforms

📚 Content Curation
Selected relevant blog articles
Created original content for top fitness topics

📱 Product Knowledge
Audited the app to identify content needs
Prioritized topics based on relevance to new users

📈 Improvements

Refining the LLM was an ongoing process that involved multiple rounds of testing and iteration. To establish a clear evaluation framework, I collaborated with the QA engineer to develop a rubric that defined key criteria for assessing the LLM’s responses. Using this framework, I conducted an initial round of internal testing. Once satisfactory results were achieved, testing was expanded to a subset of internal users to further refine the model before finally releasing to 100% of english speaking users.

Since the app is available in several languages, documentation and preprompt improvements were also made to prepare the model for multilingual deployment. This included breaking down information into smaller tokens and incorporating language specifications to handle localized names. During this phase, I supervised and trained an intern who took over the QA process, helping him refine his data reporting, documentation, and presentation skills. This allowed me to focus on trend analysis and stakeholder reporting.

By identifying recurring questions, we uncovered key user pain points, which in turn helped inform product decisions and shape future development efforts. Coach+ contributed directly to key business outcomes, with activation rates increasing by approximately 9% during A/B testing.

🧪 Internal Testing
Collaborated with QA engineer to create an evaluation rubric
Test responses against a list of frequently asked questions
Conducted internal testing before staged feature release

🔄 Continuous Refinement
Curated content based on testing feedback
Prepared and adapted content for multilingual deployment

📊 Data-Driven Decisions
Analyzed trends and reporting insights
Identified key user pain points to inform future product decisions
Increased activation rates ~9% during A/B test

🌟 Results

Through a structured and iterative approach, I successfully led the content strategy for training Coach+, positioning it as a reliable and informative fitness assistant within the Freeletics app.

• Flexible content strategy that streamlined the process of training the LLM.

• QA rubric and process to evaluate the accuracy, tone, and personalization of LLM responses.

• Product concept map that outlined key product features

• Set up for multi-language release with language-specific elements

• Information architecture that established an organized content hierarchy and product knowledge structure.

• Data and performance reports for stakeholders that provided actionable insights.

• Increased activation rate ~9% during A/B testing

• Supervised and mentored an intern in their documentation, data analysis, and presentation skills.

Coach+ launched successfully to English-speaking users with a solid foundation for expanding to additional languages. Which leaves the feature in a good state to be expanded upon and improved.

🎯 Accomplishments
Flexible content strategy
QA rubric and process
Product concept map
Set up for multi-language release
Scalable information architecture
Data and performance reports
Increased retention ~9% in A/B tests
Mentored and trained intern on data reporting skills

☀️ Wrap up
Personal lessons