extra.inpro.training
Class DataCollector

java.lang.Object
  java.awt.Component
      java.awt.Container
          javax.swing.JComponent
              javax.swing.JPanel
                  extra.inpro.training.DataCollector

All Implemented Interfaces:

java.awt.event.ActionListener, java.awt.image.ImageObserver, java.awt.MenuContainer, java.io.Serializable, java.util.EventListener, javax.accessibility.Accessible

public class DataCollector
extends javax.swing.JPanel
implements java.awt.event.ActionListener

Software to collect user-annotated semi-spontaneous speech. DataCollector combines ASR and recording tool in order to improve understanding between users and ASR. The user gets a feeling for what the ASR may be able to understand while the annotated output from the user can be used to further improve ASR.

DataCollector can be distributed as a Java WebStart application which enables data collections from a broad (and cheap) user base.

Visual cues to what should or could be recorded can be given in a slide show panel, which also enables the collector to prime specific words or contexts in which an utterance is produced.

This approach allows to balance the degree of freedom the collected data will expose: Very generic slide shows lead to almost spontaneous utterances, while very specific slides (i.e. spelling out the sentence to read) result in non-spontaneous, read data.

Author:

timo

See Also:

Serialized Form

Nested Class Summary

private class	DataCollector.RecoRunner
private class	DataCollector.RecoStart RecoStart is used to start up recognition (AM loading, ...) in a SwingWorker thread.

Nested classes/interfaces inherited from class javax.swing.JPanel

javax.swing.JPanel.AccessibleJPanel

Nested classes/interfaces inherited from class javax.swing.JComponent

javax.swing.JComponent.AccessibleJComponent

Nested classes/interfaces inherited from class java.awt.Container

java.awt.Container.AccessibleAWTContainer

Nested classes/interfaces inherited from class java.awt.Component

java.awt.Component.AccessibleAWTComponent, java.awt.Component.BaselineResizeBehavior, java.awt.Component.BltBufferStrategy, java.awt.Component.FlipBufferStrategy

Field Summary

private javax.swing.JButton	acceptButton
private CurrentHypothesisViewer	chv Sphinx ResultListener with a JTextField showing the current hypothesis
private CommandLineOptions	configuration
private javax.swing.JButton	discardButton
(package private) MetaData	metaData meta data for this recording session
private javax.swing.JButton	playbackButton
private DataCollector.RecoRunner	recoRunner encapsulates (most) ASR stuff
private javax.swing.JTextField	resultField
private SessionData	sessionData stores references to the various files, so that they can be sent to the server later on
private SlideShowPanel	slidePanel panel to show and control an inspirational slide show
private SpeechStateVisualizer	ssv
private javax.swing.JButton	uploadButton
private WavTEDLogger	wavWriter Sphinx FrontEnd processor which writes VAD speech chunks to disk

Fields inherited from class javax.swing.JComponent

accessibleContext, listenerList, TOOL_TIP_TEXT_KEY, ui, UNDEFINED_CONDITION, WHEN_ANCESTOR_OF_FOCUSED_COMPONENT, WHEN_FOCUSED, WHEN_IN_FOCUSED_WINDOW

Fields inherited from class java.awt.Component

BOTTOM_ALIGNMENT, CENTER_ALIGNMENT, LEFT_ALIGNMENT, RIGHT_ALIGNMENT, TOP_ALIGNMENT

Fields inherited from interface java.awt.image.ImageObserver

ABORT, ALLBITS, ERROR, FRAMEBITS, HEIGHT, PROPERTIES, SOMEBITS, WIDTH

Constructor Summary

DataCollector(CommandLineOptions configuration)

Method Summary

void	actionPerformed(java.awt.event.ActionEvent ae) Action handler for all actions in the application.
static void	createAndShowGUI(CommandLineOptions configuration) used to construct the GUI on the Swing thread.
(package private) static javax.swing.JButton	createButton(java.lang.String icon, java.lang.String tooltip, java.awt.event.ActionListener al, java.lang.String command) used in GUI creation
static void	main(java.lang.String[] args) main method for the application
void	playFile(java.lang.String fileName) playback the given audio file to the speakers
private void	saveTranscript(java.lang.String transcript, java.lang.String filetype) save a transcript to sessiondata
private void	setOptions() add config files to sessionData.
private void	setRecognizing(boolean recognizing)

Methods inherited from class javax.swing.JPanel

getAccessibleContext, getUI, getUIClassID, paramString, setUI, updateUI

Methods inherited from class javax.swing.JComponent

addAncestorListener, addNotify, addVetoableChangeListener, computeVisibleRect, contains, createToolTip, disable, enable, firePropertyChange, firePropertyChange, firePropertyChange, fireVetoableChange, getActionForKeyStroke, getActionMap, getAlignmentX, getAlignmentY, getAncestorListeners, getAutoscrolls, getBaseline, getBaselineResizeBehavior, getBorder, getBounds, getClientProperty, getComponentGraphics, getComponentPopupMenu, getConditionForKeyStroke, getDebugGraphicsOptions, getDefaultLocale, getFontMetrics, getGraphics, getHeight, getInheritsPopupMenu, getInputMap, getInputMap, getInputVerifier, getInsets, getInsets, getListeners, getLocation, getMaximumSize, getMinimumSize, getNextFocusableComponent, getPopupLocation, getPreferredSize, getRegisteredKeyStrokes, getRootPane, getSize, getToolTipLocation, getToolTipText, getToolTipText, getTopLevelAncestor, getTransferHandler, getVerifyInputWhenFocusTarget, getVetoableChangeListeners, getVisibleRect, getWidth, getX, getY, grabFocus, isDoubleBuffered, isLightweightComponent, isManagingFocus, isOpaque, isOptimizedDrawingEnabled, isPaintingForPrint, isPaintingTile, isRequestFocusEnabled, isValidateRoot, paint, paintBorder, paintChildren, paintComponent, paintImmediately, paintImmediately, print, printAll, printBorder, printChildren, printComponent, processComponentKeyEvent, processKeyBinding, processKeyEvent, processMouseEvent, processMouseMotionEvent, putClientProperty, registerKeyboardAction, registerKeyboardAction, removeAncestorListener, removeNotify, removeVetoableChangeListener, repaint, repaint, requestDefaultFocus, requestFocus, requestFocus, requestFocusInWindow, requestFocusInWindow, resetKeyboardActions, reshape, revalidate, scrollRectToVisible, setActionMap, setAlignmentX, setAlignmentY, setAutoscrolls, setBackground, setBorder, setComponentPopupMenu, setDebugGraphicsOptions, setDefaultLocale, setDoubleBuffered, setEnabled, setFocusTraversalKeys, setFont, setForeground, setInheritsPopupMenu, setInputMap, setInputVerifier, setMaximumSize, setMinimumSize, setNextFocusableComponent, setOpaque, setPreferredSize, setRequestFocusEnabled, setToolTipText, setTransferHandler, setUI, setVerifyInputWhenFocusTarget, setVisible, unregisterKeyboardAction, update

Methods inherited from class java.awt.Container

add, add, add, add, add, addContainerListener, addImpl, addPropertyChangeListener, addPropertyChangeListener, applyComponentOrientation, areFocusTraversalKeysSet, countComponents, deliverEvent, doLayout, findComponentAt, findComponentAt, getComponent, getComponentAt, getComponentAt, getComponentCount, getComponents, getComponentZOrder, getContainerListeners, getFocusTraversalKeys, getFocusTraversalPolicy, getLayout, getMousePosition, insets, invalidate, isAncestorOf, isFocusCycleRoot, isFocusCycleRoot, isFocusTraversalPolicyProvider, isFocusTraversalPolicySet, layout, list, list, locate, minimumSize, paintComponents, preferredSize, printComponents, processContainerEvent, processEvent, remove, remove, removeAll, removeContainerListener, setComponentZOrder, setFocusCycleRoot, setFocusTraversalPolicy, setFocusTraversalPolicyProvider, setLayout, transferFocusBackward, transferFocusDownCycle, validate, validateTree

Methods inherited from class java.awt.Component

action, add, addComponentListener, addFocusListener, addHierarchyBoundsListener, addHierarchyListener, addInputMethodListener, addKeyListener, addMouseListener, addMouseMotionListener, addMouseWheelListener, bounds, checkImage, checkImage, coalesceEvents, contains, createImage, createImage, createVolatileImage, createVolatileImage, disableEvents, dispatchEvent, enable, enableEvents, enableInputMethods, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, getBackground, getBounds, getColorModel, getComponentListeners, getComponentOrientation, getCursor, getDropTarget, getFocusCycleRootAncestor, getFocusListeners, getFocusTraversalKeysEnabled, getFont, getForeground, getGraphicsConfiguration, getHierarchyBoundsListeners, getHierarchyListeners, getIgnoreRepaint, getInputContext, getInputMethodListeners, getInputMethodRequests, getKeyListeners, getLocale, getLocation, getLocationOnScreen, getMouseListeners, getMouseMotionListeners, getMousePosition, getMouseWheelListeners, getName, getParent, getPeer, getPropertyChangeListeners, getPropertyChangeListeners, getSize, getToolkit, getTreeLock, gotFocus, handleEvent, hasFocus, hide, imageUpdate, inside, isBackgroundSet, isCursorSet, isDisplayable, isEnabled, isFocusable, isFocusOwner, isFocusTraversable, isFontSet, isForegroundSet, isLightweight, isMaximumSizeSet, isMinimumSizeSet, isPreferredSizeSet, isShowing, isValid, isVisible, keyDown, keyUp, list, list, list, location, lostFocus, mouseDown, mouseDrag, mouseEnter, mouseExit, mouseMove, mouseUp, move, nextFocus, paintAll, postEvent, prepareImage, prepareImage, processComponentEvent, processFocusEvent, processHierarchyBoundsEvent, processHierarchyEvent, processInputMethodEvent, processMouseWheelEvent, remove, removeComponentListener, removeFocusListener, removeHierarchyBoundsListener, removeHierarchyListener, removeInputMethodListener, removeKeyListener, removeMouseListener, removeMouseMotionListener, removeMouseWheelListener, removePropertyChangeListener, removePropertyChangeListener, repaint, repaint, repaint, resize, resize, setBounds, setBounds, setComponentOrientation, setCursor, setDropTarget, setFocusable, setFocusTraversalKeysEnabled, setIgnoreRepaint, setLocale, setLocation, setLocation, setName, setSize, setSize, show, show, size, toString, transferFocus, transferFocusUpCycle

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

ssv

private final SpeechStateVisualizer ssv

uploadButton

private final javax.swing.JButton uploadButton

acceptButton

private final javax.swing.JButton acceptButton

discardButton

private final javax.swing.JButton discardButton

playbackButton

private final javax.swing.JButton playbackButton

resultField

private final javax.swing.JTextField resultField

slidePanel

private final SlideShowPanel slidePanel

panel to show and control an inspirational slide show

chv

private final CurrentHypothesisViewer chv

Sphinx ResultListener with a JTextField showing the current hypothesis

wavWriter

private final WavTEDLogger wavWriter

Sphinx FrontEnd processor which writes VAD speech chunks to disk

recoRunner

private final DataCollector.RecoRunner recoRunner

encapsulates (most) ASR stuff

sessionData

private final SessionData sessionData

stores references to the various files, so that they can be sent to the server later on

metaData

MetaData metaData

meta data for this recording session

configuration

private final CommandLineOptions configuration

Constructor Detail

DataCollector

DataCollector(CommandLineOptions configuration)

Method Detail

createButton

static javax.swing.JButton createButton(java.lang.String icon,
                                        java.lang.String tooltip,
                                        java.awt.event.ActionListener al,
                                        java.lang.String command)

used in GUI creation

Parameters:

icon - name of the icon resource (relative to this class)

tooltip - tooltip for this button

al - action listener for this button

command - executed on action listener

Returns:

a new button with all the information from above

setOptions

private void setOptions()

add config files to sessionData.

setRecognizing

private void setRecognizing(boolean recognizing)

actionPerformed

public void actionPerformed(java.awt.event.ActionEvent ae)

Action handler for all actions in the application. handles all (four) actions in the GUI plus the one action from ASR.

Specified by:

actionPerformed in interface java.awt.event.ActionListener

saveTranscript

private void saveTranscript(java.lang.String transcript,
                            java.lang.String filetype)

save a transcript to sessiondata

playFile

public void playFile(java.lang.String fileName)

playback the given audio file to the speakers

createAndShowGUI

public static void createAndShowGUI(CommandLineOptions configuration)

used to construct the GUI on the Swing thread.

Parameters:

configuration - the configuration derived from the command line options

main

public static void main(java.lang.String[] args)

main method for the application

Parameters:

args - for argument parsing see CommandLineOptions