Silent Speech Interface (SSI) can enable interaction in a new and natural way based on no-audible biosignals generated by the human body. Electroencephalography (EEG) or surface electromyography (sEMG) generated during speech production can be utilized to decode silent speech. However, obtaining complementary information from EEG and sEMG is still challenging. This paper presents a hybrid SSI based on the converter between bimodal electrophysiological signals and audio signals. EEG and sEMG are fused through two sequence-to-sequence models, and multi-task losses are applied to achieve complementarity between speech intention and muscle activity in silent speech. The feasibility of the proposed fusion method is validated in the silent speech dataset, and an average objective character error rate (CER) of 7.22% among eight speakers is obtained. The experimental results show that our bimodal-based hybrid SSI facilitates the conversion of electrophysiological signals to audio.