Adding auto-generated video to your slides
OverviewQuestions:Objectives:
How can we add auto-generated video?
How does it work?
What do I need to do to make it optimal for viewers?
Adding a video to a set of slides
Time estimation: 20 minutesSupporting Materials:Last modification: Oct 18, 2022
Video Lectures
Based on the work by Delphine Larivière and James Taylor with their COVID-19 Lectures we have implemented a similar feature in the Galaxy Training Network.
AgendaIn this tutorial, we will:
How it Works
We wrote a short script which does the following:
Locally and in production:
- Extracts a ‘script’ from the slides. We extract every presenter comment in the slidedeck, and turn this into a text file.
- Every line of this text file is then narrated by Amazon Polly (if you have money) or MozillaTTS (free).
- The slide deck is converted to a PDF, and then each slide is extracted as a PNG.
- Captions are extracted from the audio components.
- The narration is stitched together into an mp3
- The images are stitched together into an mp4 file
- The video, audio, and captions are muxed together into a final mp4 file
In production
- We use Amazon Polly, paid for by the Galaxyproject
- The result is uploaded to an S3 bucket
Enabling Video
We have attempted to simplify this process as much as possible, but making good slides which work well is up to you.
Writing Good Captions
Every slide must have some narration in the presenter notes. It does not make sense for students to see a slide without commentary. For each slide, you’ll need to write presenter notes in full, but short sentences.
Sentence Structure
Use simple and uncomplex sentences whenever possible. Break up ideas into easy to digest bits. Students will be listening to this spoken and possibly reading the captions.
2021-05-01 There used to be a limit of ~120 characters per sentence, but this is no longer an issue. We now break up sentences which are too long in the captions and show them over multiple timepoints. So if you need to write a really long sentence, you can, but we still advise to simplify sentences where possible.
Captions per Slide
Every slide must have some speaker notes in this system, NO exceptions.
Punctuation
Sentences should end with punctuation like .
or ?
or even !
if you’re feeling excited.
Abbreviations
These are generally fine as-is. (e.g. e.g.
/i.e.
is fine as-is, RNA
is fine, etc.) Make sure abbreviations are all caps though.
Good This role deploys CVMFS.
“Weird” Names
In the captions you will want to teach the GTN how to pronounce these words by editing bin/ari-map.yml
to provide your definition.
E.g.
Word | Pronunciation |
---|---|
SQLAlchemy | SQL alchemy |
FastQC | fast QC |
nginx | engine X |
gxadmin | GX admin |
/etc | / E T C |
The same applies to the many terms we read differently from how they are written, e.g. ‘src’ vs ‘source’. Most of us would pronounce it like the latter, even though it isn’t spelt that way. Our speaking robot doesn’t know what we mean, so we need to spell it out properly.
So we write the definition in the bin/ari-map.yml
file.
Other Considerations
(Written 2020-12-16, things may have changed since.)
Be sure to check the pronunciation of the slides. There are known issues with heteronyms, words spelt the same but having different pronunciation and meaning. Consider “read” for a classic example, or “analyses” for one that comes up often in the GTN. “She analyses data” and “Multiple analyses” are pronounced quite differently based on their usage in sentences. See the wiktionary page for more information, or the list of English heteronyms you might want to be aware of.
This becomes an issue for AWS Polly and Mozilla’s TTS which both don’t have sufficient context sometimes to choose between the two pronunciations. You’ll find that “many analyses” is pronounced correctly while “multiple analyses” isn’t.
Oftentimes the services don’t understand part of speech, so by adding adjectives to analyses, you confuse the engine in to thinking it should be the third person singular pronunciation. This is probably because it only has one or two words of context ahead of the word to be pronounced.
Enable the Video
Lastly, we need to tell the GTN framework we would like videos to be generated.
Hands-on: Enable video
- Edit the
slides.html
for your tutorial- Add
video: true
to the top
That’s it! With this, videos can be automatically generated.
How it works: In Detail
- We take our markdown slides, e.g.
topics/introduction/tutorials/galaxy-intro-short/slides.html
- In order for them to be processed, slides must have an annotation saying
video: true
in the header metadata, and then ‘speaker notes’ (everything after the ??? before the —) - This is turned into our ‘plain text slides’ which just renders the markdown a bit more nicely (example)
-
Then we run ari.sh which does the following:
make video
is run which runsbin/ari-make.sh
- This builds PDFs for any slides which have changed
- And runs
./bin/ari.sh
with the PDF, the original Slides, and where the mp4 should be saved.- In
./bin/ari.sh
- It extracts metadata from the tutorial (title, authors, etc.)
- It builds a ‘script’, a json document with blocks for every line of the speaker notes that were in the slides.
- Those get converted into mp3 files by AWS Polly (or MozillaTTS), one per slide.
- The PDFs get turned into a series of PNG images
- We take the timings of the mp3 files together with the json ‘script’ to write out webvtt / srt subtitles which get embedded into the video, and supplied next to it.
- editly is used to knit together the PNGs + mp3s with appropriate delay
- In
All of this is run on cron by .github/workflows/video.yml
which handles building all of these videos and then later uploading them to s3.
Many of the scripts internally are prefixed with ari
, we named our internal version after github.com/jhudsl/ari/ which inspired it, but we wanted a version that would be more closely tied to the GTN and integrate with our infrastructure nicely, so we ended up writing our own.
Conclusion
Key points
Thanks to the GTN, videos are easy to add
Be mindful of your captions. Short sentences are good!
Frequently Asked Questions
Have questions about this tutorial? Check out the tutorial FAQ page or the FAQ page for the Contributing to the Galaxy Training Material topic to see if your question is listed there. If not, please ask your question on the GTN Gitter Channel or the Galaxy Help ForumFeedback
Did you use this material as an instructor? Feel free to give us feedback on how it went.
Did you use this material as a learner or student? Click the form below to leave feedback.
Citing this Tutorial
- Helena Rasche, 2022 Adding auto-generated video to your slides (Galaxy Training Materials). https://training.galaxyproject.org/training-material/topics/contributing/tutorials/slides-with-video/tutorial.html Online; accessed TODAY
- Batut et al., 2018 Community-Driven Data Analysis Training for Biology Cell Systems 10.1016/j.cels.2018.05.012
Congratulations on successfully completing this tutorial!@misc{contributing-slides-with-video, author = "Helena Rasche", title = "Adding auto-generated video to your slides (Galaxy Training Materials)", year = "2022", month = "10", day = "18" url = "\url{https://training.galaxyproject.org/training-material/topics/contributing/tutorials/slides-with-video/tutorial.html}", note = "[Online; accessed TODAY]" } @article{Batut_2018, doi = {10.1016/j.cels.2018.05.012}, url = {https://doi.org/10.1016%2Fj.cels.2018.05.012}, year = 2018, month = {jun}, publisher = {Elsevier {BV}}, volume = {6}, number = {6}, pages = {752--758.e1}, author = {B{\'{e}}r{\'{e}}nice Batut and Saskia Hiltemann and Andrea Bagnacani and Dannon Baker and Vivek Bhardwaj and Clemens Blank and Anthony Bretaudeau and Loraine Brillet-Gu{\'{e}}guen and Martin {\v{C}}ech and John Chilton and Dave Clements and Olivia Doppelt-Azeroual and Anika Erxleben and Mallory Ann Freeberg and Simon Gladman and Youri Hoogstrate and Hans-Rudolf Hotz and Torsten Houwaart and Pratik Jagtap and Delphine Larivi{\`{e}}re and Gildas Le Corguill{\'{e}} and Thomas Manke and Fabien Mareuil and Fidel Ram{\'{\i}}rez and Devon Ryan and Florian Christoph Sigloch and Nicola Soranzo and Joachim Wolff and Pavankumar Videm and Markus Wolfien and Aisanjiang Wubuli and Dilmurat Yusuf and James Taylor and Rolf Backofen and Anton Nekrutenko and Björn Grüning}, title = {Community-Driven Data Analysis Training for Biology}, journal = {Cell Systems} }
Developing GTN training materialThis tutorial is part of a series to develop GTN training material, feel free to also look at:
- Overview of the Galaxy Training Material
- Adding auto-generated video to your slides
- Adding Quizzes to your Tutorial
- Contributing with GitHub via command-line
- Contributing with GitHub via its interface
- Creating a new tutorial
- Creating content in Markdown
- Creating Interactive Galaxy Tours
- Creating Slides
- Design and plan session, course, materials
- Generating PDF artefacts of the website
- GTN Metadata
- Including a new topic
- Principles of learning and how they apply to training and teaching
- Running the GTN website locally
- Running the GTN website online using GitPod
- Teaching Python
- Tools, Data, and Workflows for tutorials
- Updating diffs in admin training