LiveSubtitlesSRS

Introduction

Purpose of the requirements document

This Software Requirements Specification (SRS) identifies the requirements for project "Sign2Speech". This is an open source projet and we shall present what we did for this project in case to catch interest of new potential contributors. This document is a guideline about the functionalities offered and the problems that the system solve.

Scope of the product

RealTimeSubtiltes is an app designed to help partially deaf stutents in a classroom. The aim is to transcript a teacher speech in live and display the speech on the corresponding slide as subtitles. On the other hands, students in the classroom can correct the subtitle on a collaborative HMI. We have to use GoogleAPI Speech for the transcript, reveal.js for the slides and JavaScript. .

General Description

Product perspective

The main target of our project is to help partially deaf student to be more autonomous attending a lecture. This project is proposed by the department of disabled students at the UGA. In addition, we have to design a collaborative HMI for students to correct in real time the subtitles.

Product functions

The part have 2 parts :

• The transcript by GoogleSpeech

In a first place the API must recognize the teacher speech and transcript it in real time. Final result are appended into the right place according to the current slide.

• The collaborative HMI

Designed for students, it allows logged in student to follow a course. While the teacher speech the students can either follow the courses and read the subtitles, or edit the subtitles to correct the results.

User characteristics

There are three types of users for our app

• The teacher talking while showing his slides
• The students editing notes
• The students reading the notes and the partially deaf students

Operating environment

The GoogleSpeech API works on google Chrome. A good Internet connection is required for the transcript.

General constraints

• The teacher needs to have his slides on reveal.js
• The teacher need to talk loud and not so fast
• The room has to be quiet (no noise)
• These elements can reduce errors and help the API to transcript well the speech. However, it won’t be perfect due to the instability of GoogleSpeech API.

Specific requirements, covering functional, non-functional and interface requirements

Requirement X.Y.Z (in Structured Natural Language)

Speech recognition

Description: Capture the voice and return a textual translation Inputs: Voice of a speaker Source: Human Outputs: Textual data Destination: User Action: A speaker talk with a microphone and the system return the transcript in textual Non functional requirements: Accurate detection of spoken words Pre-condition: User has a microphone Post-condition: Words are detected Side-effects: words are not detected or wrong detection

Render the subtitles to slides

Description: Show the subtitles to the slides Inputs: words spoken Source: Speech recognizer Outputs: slides with subtitles Destination: slides Action: : get the spoken words and show them correctly to the slides Non functional requirements: No loss of data Pre-condition: Spoken words are detected Post-condition: Slides are shown with subtitles Side-effects:Subtitles are not well shown and hide the slides. Subtitles are not readable.

Learning mode

Description: The function of this mode is to allow the user to add as many gestures (with their translations) as he wants to the dictionary

Inputs: Hand and finger data returned by the camera stream and the meaning of the gesture

Source: Intel's Real Sense camera

Outputs: New dictionary (JSON file) containing the new gestures and their meaning

Destination: Computer's memory

Action: The user has to select the learning mode when launching the application and enter the number of gestures that he wants to record. Guidelines are printed on the screen so that the user knows what to do and when to do it. Basically, he will have to repeat each gesture 3 times in a row (so that the program can compute the average of the 3 repeated gestures to minimize the errors). At the end of the record, the user can chooose whether he wants to add a new word (and stay in the learning mode) or not (in this case, the normal recognition mode will be activated).

Non functional requirements: Real-time tracking (< 1 second)

Pre-condition: Optimal conditions of use (good light, monochrom top that constrats with the color of the skin, no rings, no bracelets, ...). The user must also have divided his gesture into basic gestures to record.

Post-condition: The new entry in the dictionary must correspond to the gesture that the user intended to do

Side-effects: The lack of precision of the camera: if the gesture was not well recognized, the encoding in the dictionary will be wrong

Product Evolution

• “Real-time” windows that could show a representation of the hand that the camera is currently analyzing. It could allow the user to know if the camera is able to correctly recognize his hand. It could be done with QT Creator. Our application is not at this time really “friendly-user”.

• “2 hands” symbols that are currently not implemented in our application

• Improvements of trajectories recognition

• Language Model

• A better camera