11-07, 11:40–12:20 (US/Eastern), Central Park East
In an era of information overload, the ability to distill vast amounts of text into clear, concise summaries is more essential than ever. Large Language Models (LLMs) have become powerful tools for this task, revolutionizing how we process and comprehend data. However, the journey from raw data to insightful summaries isn't without its hurdles. Beyond their impressive capabilities, LLMs face significant challenges, particularly when dealing with extensive texts. How can we overcome the constraints of limited context windows? How to even scale such LLMs for thousands of concurrent users? In this talk Aarti will walk through the art and science of tackling these issues, from effectively chunking long documents, optimizing model performance within limited context windows and eventually deploying such models into production. She will also demonstrate advanced evaluation techniques to ensure that the summaries generated are both accurate and meaningful. An AI enthusiast or a seasoned professional, this session will equip the attendees with strategies to enhance their text summarization and maximize the potential of LLMs.
As Large Language Models continue to advance, their application in text summarization presents both powerful opportunities and specific challenges. This talk will focus on practical strategies to overcome the limitations posed by context windows—a critical factor when dealing with extensive texts. The talk will also demonstrate how fine-tuning can improve summarization tasks for domain specific private datasets and when to use what. Attendees will learn how to build an end-to-end summarization workflow, with a focus on effective data chunking, prompt optimization, and advanced evaluation methods to ensure accurate and meaningful summaries.
The session will cover three key summarization techniques—stuff, refine, and map-reduce—explaining when and how to use each approach. In addition, we’ll explore the latest in evaluation metrics, demonstrating how to leverage more sophisticated models as judges to refine and assess the quality of summaries.
Outline:
* Introduction to the summarization problem
* Understanding LLM context windows
* Chunking techniques and their applications
* Overview of summarization techniques (stuff, refine, map-reduce)
* Fine-tuning on a custom dataset
* Introduction to Langchain and its role in summarization
* Evaluation metrics
* Using LLMs as judges
* LLM production deployment
Background Knowledge Required:
* Basic understanding of LLMs
* Familiarity with Python
Presentation - https://github.com/aartij22/Pydata-NYC-2024
Previous knowledge expected
Aarti Jha is currently a Senior Data Scientist at Red Hat, Bengaluru, India, where she develops AI-driven solutions to streamline processes and reduce operational costs for internal initiatives. With over six years of experience, she has previously led the development of search and recommendation systems for e-pharma at her prior organisation.
In her free time, Aarti enjoys bringing her creative visions to life through sketching.