The paper describes our initial effort to use Transformer-based neural networks for understanding and presenting oral history archives. Such archives of interviews often contain large passages of the interviewee's speech. Our approach automatically generates relevant questions, which enrich such monotonous parts and allows the listener to better orient in the interview. The generated questions also allow for finding interesting parts of the interview without changing the original meaning of the testimony. We present our working pipeline consisting of a Wav2Vec speech recognizer, BERT-based punctuation detection, T5 asking questions model and BERT-based semantic continuity model.