CLIENT
The Associated Press (AP) is an independent global news organisation renowned for its commitment to factual reporting. Founded in 1846, AP remains a trusted source of fast, accurate, and unbiased news in all formats, reaching over half the world’s population daily. As a leader in the news industry, AP looks for ways to carefully deploy artificial intelligence in areas where they can be more efficient and effective, including news gathering, the production process and news distribution. You can learn more about AP's AI innovations here.
CHALLENGE
Optimising News Summaries using the summarisation abilities of LLMs to generate accurate, concise draft summaries to serve as starting points for editors, allowing them to focus more on improving the quality and consistency of the content. This use case was the most challenging, requiring Pluralit to meet AP’s extensive requirements for accuracy and style, and overcome weaknesses of LLMs such as temporal understanding, and hallucinations.
HOW THE PLURALIT TEAM WORKED WITH AP
-
Collaborative Development: We worked closely with AP’s editorial, technical, and product teams to align solutions with their goals.
-
Agile Methodology: We adopted weekly sprints to test, refine, and deploy AI capabilities rapidly.
-
PoC/ Pilot/ Pre-Production: We successfully delivered all 3 use cases through to pre-production, using AP’s development framework and meeting gate criteria for progression from one stage to the next.
-
Feedback-Driven Improvements: We leveraged real-time user insights to enhance functionality and user satisfaction.
-
Transparent Integration: We adopted AP’s development and communication practices to act as an integrated partner to their editorial, engineering and data science teams.
SOLUTION
-
Iterative Prompt Refinement: Extensive comparative testing of LLMs including Claude, DeepL, and several OpenAI models to ensure focused, high-quality summaries. Initial prompts were refined based on editorial feedback and evaluation data to improve quality and eliminate hallucinations.
-
Guardrails AI: Hallucination detection using guardrails. For this, overlapping chunks weighted with NER (Name Entity Recognition) were used to accurately detect hallucinations.
-
Improving Speed Performance: Implementing parallelisation and MapReduce to process chunks for hallucination detection in parallel to reduce the detection time.
-
Measurement and feedback tooling: Tools to gather data, track editorial inputs, and monitor ‘editing distance’ - the effort required by editors to finalise a published summary from the AI-generated summary.
-
Fine-tuning and model optimisation: Unique challenges, such as handling temporal expressions (e.g., “yesterday” vs. absolute dates) and minimising hallucinations, were addressed by creating conversational flow examples and training the LLM through fine-tuning, as well as adding temporal context to the agent by adding relative date information.
RESULTS AND BENEFITS
The success of all these use cases, from idea through PoC and Pilot to Pre-Production, delivered some transformative outcomes for AP:
-
Increased Efficiency: Accelerated workflows across news summaries, enabled faster and more reliable news production so that AP can produce even more high-quality news output.
-
Improved Editorial Focus: Freed up editors to concentrate on refining great content and ensuring quality, rather than starting from scratch.
-
Enhanced Creativity: Provided editors with innovative, AI-generated summaries, enriching the creative process.
-
Seamless Integration: Deployed scalable APIs and interfaces that aligned with AP’s existing systems, ensuring minimal disruption and easy adoption into production.
TRANSFORM YOUR WORKFLOWS WITH PLURALIT
The collaboration between AP and Pluralit demonstrates the transformative potential of Generative AI in modern journalism. By integrating advanced technology with editorial expertise, AP has set a new standard for innovation in the media industry.
Ready to redefine your workflows? Contact us to explore how our GenAI solutions can enhance creativity, drive efficiency, and deliver measurable results for your organisation.