🧩 The problem

Summarization is a useful process to condense information while retaining the original message. There are several ways to perform this, but, of course, AI is the trend now. If you follow my YouTube channel you know I tend to prefer self-hosted solutions. These systems guarantee more privacy and independence, but there are some trade offs.

In this post, and subsequent ones, I'll show you some tests I made for summarization purposes using the Hugging Face Transformers Python library, locally, on CPU.

🤔 Reason

My final objective is to automatically generate YouTube video summaries to put in blog posts.

Now, I can't copy the YouTube video descriptions verbatim for SEO purposes: search engines don't like duplicate content. The obvious solution is to do some kind of summarization. First of all, I always use whisper and proofread the SRT file containing the recognized audio before uploading it to YouTube. I then copy the SRT file verbatim in an LLM chat, in chunks, and ask ChatGPT to generate a summary once I say so. This last step is tedious and you always rely on a SaaS. Some alternatives include the use local AI models via Ollama, or more specialized software that can run different tasks besides text generation.

The Python Transformers library by Hugging Face does just that and, thanks to its Pipeline API it is trivial to do this work (or it should be ?!).

🤗 Hugging face documentation

If you search the official docs for summarization, this is the first page that comes up. The TL;DR of that page is:

• fine-tune a model
• perform inference to generate a summary

To keep things simple, I just wanted to implement use a pre-trained model for inference.

🖥️ Meet the pipeline API

In HF, pipelines are probably the simplest notion I learned until now. If you follow this other tutorial you'll see that you can perform most tasks, including summarization, with just 3 lines of code. So this is more or less what I tried first (I added newlines to avoid scrolling)¹:

The result is this²:

The first two lines are warnings from the Transformers library which can be suppressed by setting a less verbose logging level.

As I said, I'm doing inference on CPU, so no surprises here. About the max_length parameter, it corresponds to the maximum number of tokens in the output. Apparently the default value for this model is too big for this input: it should be tweaked appropriately so that the summary still makes sense.

🎉 Conclusion

Can you see the potential problem here? Although the quality of the summary seems OK for this specific text, the model seems to have been trained for bigger inputs. In-fact, the length of the summary is about 82% of length of the input text.

In the next posts we'll see how to improve this kind of inference. I also tested other models to see how they measure up, so stay tuned.

Footnotes

Code

---

1. 
   from transformers import pipeline
   
   pipeline = pipeline(task='summarization', model='facebook/bart-large-cnn')
   
   # Text from https://en.wikipedia.org/wiki/Artificial_intelligence
   # Creative Commons Attribution-ShareAlike 4.0 License
   print(pipeline('Artificial intelligence (AI) is the capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making. It is a field of research in computer science that develops and studies methods and software that enable machines to perceive their environment and use learning and intelligence to take actions that maximize their chances of achieving  defined goals')[0]['summary_text'])

2. 
   
   Device set to use cpu ↩
   
   Your max_length is set to 142, but your input_length is only 79. Since this is a summarization task, where outputs shorter than the input are typically wanted, you might consider decreasing max_length manually, e.g. summarizer('...', max_length=39)
   
   Artificial intelligence is the capability of computational systems to perform tasks typically associated with human intelligence. It is a field of research in computer science that develops and studies methods and software that enable machines to perceive their environment and use learning and intelligence to take actions that maximize their chances of achieving defined goals.