1
0
mirror of https://github.com/signalwire/freeswitch.git synced 2025-04-18 01:28:42 +00:00
Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Fri Jun 26 18:17:10 2009 +0000

    Implemented START-INPUT-TIMERS

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1014 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Fri Jun 26 16:56:38 2009 +0000

    Fixed grammar file creation (the existing file must be truncated/overridden)

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1013 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Fri Jun 26 15:31:47 2009 +0000

    Added GET-RESULT processing, properly set completion causes in the responses to DEFINE-GRAMMAR and RECOGNIZE requests

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1012 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Thu Jun 25 19:21:34 2009 +0000

    Implemented grammar load/unload (DEFINE-GRAMMAR requests), implemented recognition complete timeout, introduced properties (to be loaded from config file)

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1011 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Wed Jun 24 14:31:37 2009 +0000

    Added the NOTICE corresponding to the section 4 d of the Apache License, Version 2.0

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1010 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Tue Jun 23 21:04:44 2009 +0000

    Added identifier@pocketsphinx traces

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1009 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Tue Jun 23 21:03:02 2009 +0000

   Added session identifier to engine channel (to be used for traces)

   git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1008 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Tue Jun 23 19:58:48 2009 +0000

   Properly handled race between STOP request and RECOGNITION-COMPLETE event, other enhancements

   git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1007 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Tue Jun 23 19:55:57 2009 +0000

   Added mpf_activity_detector_reset()

   git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1006 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Tue Jun 23 16:41:51 2009 +0000

   Added 32kHz and 48kHz sampling rates (still no resampling support in MPF, also make sure RTP packet length <= 1500 bytes)

   git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1005 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Mon Jun 22 20:13:29 2009 +0000

   Added flite m4 configure stuff

   git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1004 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Mon Jun 22 19:31:32 2009 +0000

    Added pocketsphinx.vcproj and flite.vcproj project files into unimrcp.sln (disabled by default)

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1003 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Mon Jun 22 19:26:38 2009 +0000

    Integrated utterance detection stuff (start-of-input, end-of-input), triggered partial and final recognition results

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1002 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: garmt.noname@gmail.com <garmt.noname@gmail.com@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Sun Jun 21 22:12:37 2009 +0000

    First implementation of flite plugin - voice awb only

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1001 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Sun Jun 21 06:23:58 2009 +0000

    Added framework for the development of flite plugin

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@1000 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Sat Jun 20 19:52:22 2009 +0000

    Added missing include paths and libs required for pocketsphinx

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@999 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Sat Jun 20 17:32:37 2009 +0000

    Started actual implementation of pocketsphinx plugin

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@998 f001bc3a-424a-0410-80a0-a715b8f413a8

Author: achaloyan <achaloyan@f001bc3a-424a-0410-80a0-a715b8f413a8>
Date:   Sat Jun 20 13:55:41 2009 +0000

    Added acmacros for pocketsphinx libs (src and installed dirs are suppored), cleaned up ./configure related stuff a bit

    git-svn-id: https://unimrcp.googlecode.com/svn/trunk@997 f001bc3a-424a-0410-80a0-a715b8f413a8





git-svn-id: http://svn.freeswitch.org/svn/freeswitch/trunk@13987 d0543943-73ff-0310-b7d9-9358b9ac24b2
This commit is contained in:
Michael Jerris 2009-06-26 20:18:39 +00:00
parent fd8106afcf
commit ba1aee8e3c
25 changed files with 1999 additions and 85 deletions

@ -1 +1 @@
Fri Jun 19 22:39:34 CDT 2009
Fri Jun 26 15:16:40 CDT 2009

47
libs/unimrcp/NOTICE Normal file

@ -0,0 +1,47 @@
The UniMRCP Project (http://www.unimrcp.org)
Copyright (C) 2008 Arsen Chaloyan
Licensed under the Apache License, Version 2.0 (the "License").
This product includes a number of subcomponents with
separate copyright notices and license terms.
Notice for APR (Apache Portable Runtime) library
---------------------------------------------------
This product includes software developed by
The Apache Software Foundation (http://www.apache.org/).
Portions of this software were developed at the National Center
for Supercomputing Applications (NCSA) at the University of
Illinois at Urbana-Champaign.
This software contains code derived from the RSA Data Security
Inc. MD5 Message-Digest Algorithm.
---------------------------------------------------
Notice for Sofia-SIP library
---------------------------------------------------
Copyright (C) 2005-2006 Nokia Corporation and others (see the
in individual files for a detailed list of copyright holders).
Contact: Pekka Pessi <Pekka.Pessi@nokia.com>
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public License
as published by the Free Software Foundation; either version 2.1 of
the License, or (at your option) any later version.
---------------------------------------------------
Notice for G711 implementation
---------------------------------------------------
g711.h/g711.c - A-law and u-law transcoding routines
Written by Steve Underwood <steveu@coppice.org>
Copyright (C) 2006 Steve Underwood
Despite my general liking of the GPL, I place this code in the
public domain for the benefit of all mankind - even the slimy
ones who might try to proprietize my work and use it to my
detriment.
---------------------------------------------------

@ -4,3 +4,6 @@ m4_include([build/acmacros/find_apr.m4])
m4_include([build/acmacros/find_apu.m4])
m4_include([build/acmacros/sofia-sip.m4])
m4_include([build/acmacros/swift.m4])
m4_include([build/acmacros/sphinxbase.m4])
m4_include([build/acmacros/pocketsphinx.m4])
m4_include([build/acmacros/flite.m4])

@ -0,0 +1,48 @@
dnl UNIMRCP_CHECK_FLITE
AC_DEFUN([UNIMRCP_CHECK_FLITE],
[
AC_MSG_NOTICE([Flite library configuration])
AC_MSG_CHECKING([for Flite])
AC_ARG_WITH(flite,
[ --with-flite=PATH prefix for installed Flite or
path to Flite build tree],
[flite_path=$withval],
[flite_path="/usr/local"]
)
found_flite="no"
dnl TO BE DONE
flite_libdir="build/libs"
for dir in $flite_path ; do
cd $dir && flite_dir=`pwd` && cd - > /dev/null
if test -d "$dir/$flite_libdir"; then
found_flite="yes"
UNIMRCP_FLITE_INCLUDES="-I$flite_dir/include"
UNIMRCP_FLITE_LIBS="$dir/$flite_libdir/libflite_cmu_us_awb.a \
$dir/$flite_libdir/libflite_cmu_us_kal.a \
$dir/$flite_libdir/libflite_cmu_us_rms.a \
$dir/$flite_libdir/libflite_cmu_us_slt.a \
$dir/$flite_libdir/libflite_cmulex.a \
$dir/$flite_libdir/libflite_usenglish.a \
$dir/$flite_libdir/libflite.a"
break
fi
done
if test x_$found_flite != x_yes; then
AC_MSG_ERROR(Cannot find Flite - looked for srcdir:$flite_srcdir in $flite_path)
else
AC_MSG_RESULT([$found_flite])
case "$host" in
*darwin*)
UNIMRCP_FLITE_LIBS="$UNIMRCP_FLITE_LIBS -framework CoreFoundation -framework SystemConfiguration" ;;
esac
AC_SUBST(UNIMRCP_FLITE_INCLUDES)
AC_SUBST(UNIMRCP_FLITE_LIBS)
fi
])

@ -0,0 +1,50 @@
dnl UNIMRCP_CHECK_POCKETSPHINX
AC_DEFUN([UNIMRCP_CHECK_POCKETSPHINX],
[
AC_MSG_NOTICE([PocketSphinx library configuration])
AC_MSG_CHECKING([for PocketSphinx])
AC_ARG_WITH(pocketsphinx,
[ --with-pocketsphinx=PATH prefix for installed PocketSphinx or
path to PocketSphinx build tree],
[pocketsphinx_path=$withval],
[pocketsphinx_path="/usr/local"]
)
found_pocketsphinx="no"
pocketsphinx_config="lib/pkgconfig/pocketsphinx.pc"
pocketsphinx_srcdir="src"
for dir in $pocketsphinx_path ; do
cd $dir && pocketsphinx_dir=`pwd` && cd - > /dev/null
if test -f "$dir/$pocketsphinx_config"; then
found_pocketsphinx="yes"
UNIMRCP_POCKETSPHINX_INCLUDES="`pkg-config --cflags $dir/$pocketsphinx_config`"
UNIMRCP_POCKETSPHINX_LIBS="`pkg-config --libs $dir/$pocketsphinx_config`"
pocketsphinx_version="`pkg-config --modversion $dir/$pocketsphinx_config`"
break
fi
if test -d "$dir/$pocketsphinx_srcdir"; then
found_pocketsphinx="yes"
UNIMRCP_POCKETSPHINX_INCLUDES="-I$pocketsphinx_dir/include"
UNIMRCP_POCKETSPHINX_LIBS="$pocketsphinx_dir/$pocketsphinx_srcdir/libpocketsphinx/libpocketsphinx.la"
pocketsphinx_version="`pkg-config --modversion $pocketsphinx_dir/pocketsphinx.pc`"
break
fi
done
if test x_$found_pocketsphinx != x_yes; then
AC_MSG_ERROR(Cannot find PocketSphinx - looked for pocketsphinx-config:$pocketsphinx_config and srcdir:$pocketsphinx_srcdir in $pocketsphinx_path)
else
AC_MSG_RESULT([$found_pocketsphinx])
AC_MSG_RESULT([$pocketsphinx_version])
case "$host" in
*darwin*)
UNIMRCP_POCKETSPHINX_LIBS="$UNIMRCP_POCKETSPHINX_LIBS -framework CoreFoundation -framework SystemConfiguration" ;;
esac
AC_SUBST(UNIMRCP_POCKETSPHINX_INCLUDES)
AC_SUBST(UNIMRCP_POCKETSPHINX_LIBS)
fi
])

@ -12,6 +12,7 @@ AC_DEFUN([UNIMRCP_CHECK_SOFIA],
[sofia_path="/usr/local"]
)
found_sofia="no"
sofiaconfig="lib/pkgconfig/sofia-sip-ua.pc"
sofiasrcdir="libsofia-sip-ua"
for dir in $sofia_path ; do

@ -0,0 +1,50 @@
dnl UNIMRCP_CHECK_SPHINXBASE
AC_DEFUN([UNIMRCP_CHECK_SPHINXBASE],
[
AC_MSG_NOTICE([SphinxBase library configuration])
AC_MSG_CHECKING([for SphinxBase])
AC_ARG_WITH(sphinxbase,
[ --with-sphinxbase=PATH prefix for installed SphinxBase or
path to SphinxBase build tree],
[sphinxbase_path=$withval],
[sphinxbase_path="/usr/local"]
)
found_sphinxbase="no"
sphinxbase_config="lib/pkgconfig/sphinxbase.pc"
sphinxbase_srcdir="src"
for dir in $sphinxbase_path ; do
cd $dir && sphinxbase_dir=`pwd` && cd - > /dev/null
if test -f "$dir/$sphinxbase_config"; then
found_sphinxbase="yes"
UNIMRCP_SPHINXBASE_INCLUDES="`pkg-config --cflags $dir/$sphinxbase_config`"
UNIMRCP_SPHINXBASE_LIBS="`pkg-config --libs $dir/$sphinxbase_config`"
sphinxbase_version="`pkg-config --modversion $dir/$sphinxbase_config`"
break
fi
if test -d "$dir/$sphinxbase_srcdir"; then
found_sphinxbase="yes"
UNIMRCP_SPHINXBASE_INCLUDES="-I$sphinxbase_dir/include"
UNIMRCP_SPHINXBASE_LIBS="$sphinxbase_dir/$sphinxbase_srcdir/libsphinxbase/libsphinxbase.la $sphinxbase_dir/$sphinxbase_srcdir/libsphinxad/libsphinxad.la"
sphinxbase_version="`pkg-config --modversion $sphinxbase_dir/sphinxbase.pc`"
break
fi
done
if test x_$found_sphinxbase != x_yes; then
AC_MSG_ERROR(Cannot find SphinxBase - looked for sphinxbase-config:$sphinxbase_config and srcdir:$sphinxbase_srcdir in $sphinxbase_path)
else
AC_MSG_RESULT([$found_sphinxbase])
AC_MSG_RESULT([$sphinxbase_version])
case "$host" in
*darwin*)
UNIMRCP_SPHINXBASE_LIBS="$UNIMRCP_SPHINXBASE_LIBS -framework CoreFoundation -framework SystemConfiguration" ;;
esac
AC_SUBST(UNIMRCP_SPHINXBASE_INCLUDES)
AC_SUBST(UNIMRCP_SPHINXBASE_LIBS)
fi
])

@ -25,6 +25,4 @@ AC_DEFUN([UNIMRCP_CHECK_SWIFT],
else
AC_MSG_WARN([not found - looked for $swift_path])
fi
AM_CONDITIONAL([CEPSTRAL_PLUGIN], [test x_$found_swift = x_yes])
])

@ -0,0 +1,22 @@
<?xml version="1.0" encoding="windows-1251"?>
<VisualStudioPropertySheet
ProjectType="Visual C++"
Version="8.00"
Name="flite"
InheritedPropertySheets=".\unibase.vsprops"
>
<Tool
Name="VCCLCompilerTool"
AdditionalIncludeDirectories="&quot;$(FliteDir)\include&quot;"
PreprocessorDefinitions="APT_LIB_EXPORT"
/>
<Tool
Name="VCLinkerTool"
AdditionalDependencies="flite.lib"
AdditionalLibraryDirectories="&quot;$(FliteDir)\$(ConfigurationName)&quot;"
/>
<UserMacro
Name="FliteDir"
Value="$(LibRootDir)libs\flite"
/>
</VisualStudioPropertySheet>

@ -16,19 +16,19 @@ datadir='${prefix}/data'
AM_INIT_AUTOMAKE(foreign)
#Set default language
# Set default language
AC_LANG_C
AC_PROG_CC
AC_PROG_CXX
AC_PROG_INSTALL
#skip detection of Fortran
# Skip detection of Fortran
m4_undefine([AC_PROG_F77])
m4_defun([AC_PROG_F77],[])
AC_PROG_LIBTOOL
# get version information
# Get version information
get_version="build/get-version.sh"
version_hdr="build/uni_version.h"
plugin_version_hdr="libs/mrcp-engine/include/mrcp_resource_plugin.h"
@ -42,12 +42,13 @@ AC_SUBST(PLUGIN_LT_VERSION)
echo "UniMRCP Version: ${UNI_DOTTED_VERSION}"
# Check APR
UNIMRCP_CHECK_APR
UNIMRCP_CHECK_APU
# Check SOFIA
UNIMRCP_CHECK_SOFIA
UNIMRCP_CHECK_SWIFT
AC_SUBST(ac_aux_dir)
@ -74,7 +75,7 @@ AC_CACHE_CHECK([for _AC_LANG compiler vendor], ax_cv_[]_AC_LANG_ABBREV[]_compile
AX_COMPILER_VENDOR
#Enable maintainer mode
AC_ARG_ENABLE(maintainer-mode,
[AC_HELP_STRING([--enable-maintainer-mode ],[turn on debugging and compile time warnings])],
[enable_maintainer_mode="$enableval"],
@ -87,6 +88,7 @@ if test "${enable_maintainer_mode}" != "no"; then
fi
fi
#Enable test suites
AC_ARG_ENABLE(test-suites,
[AC_HELP_STRING([--enable-test-suites ],[build test suites])],
[enable_test_suites="$enableval"],
@ -94,6 +96,10 @@ AC_ARG_ENABLE(test-suites,
AM_CONDITIONAL([TEST_SUITES],[test "${enable_test_suites}" != "no"])
### Plugins ###
#Enable demo synthesizer plugin
AC_ARG_ENABLE(demosynth-plugin,
[AC_HELP_STRING([--disable-demosynth-plugin ],[exclude demo synthesizer plugin from build])],
[enable_demosynth_plugin="$enableval"],
@ -101,6 +107,8 @@ AC_ARG_ENABLE(demosynth-plugin,
AM_CONDITIONAL([DEMOSYNTH_PLUGIN],[test "${enable_demosynth_plugin}" = "yes"])
#Enable demo recognizer plugin
AC_ARG_ENABLE(demorecog-plugin,
[AC_HELP_STRING([--disable-demorecog-plugin ],[exclude demo recognizer plugin from build])],
[enable_demorecog_plugin="$enableval"],
@ -108,6 +116,51 @@ AC_ARG_ENABLE(demorecog-plugin,
AM_CONDITIONAL([DEMORECOG_PLUGIN],[test "${enable_demorecog_plugin}" = "yes"])
#Enable Cepstral Swift plugin
AC_ARG_ENABLE(cepstral-plugin,
[AC_HELP_STRING([--disable-cepstral-plugin ],[exclude cepstral plugin from build])],
[enable_cepstral_plugin="$enableval"],
[enable_cepstral_plugin="yes"])
if test "${enable_cepstral_plugin}" != "no"; then
UNIMRCP_CHECK_SWIFT
fi
AM_CONDITIONAL([CEPSTRAL_PLUGIN],[test "${enable_cepstral_plugin}" = "yes" &&\
test "${found_swift}" = "yes"])
#Enable PocketSphinx plugin
AC_ARG_ENABLE(pocketsphinx-plugin,
[AC_HELP_STRING([--enable-pocketsphinx-plugin ],[enable pocketsphinx plugin])],
[enable_pocketsphinx_plugin="$enableval"],
[enable_pocketsphinx_plugin="no"])
if test "${enable_pocketsphinx_plugin}" != "no"; then
UNIMRCP_CHECK_SPHINXBASE
UNIMRCP_CHECK_POCKETSPHINX
fi
AM_CONDITIONAL([POCKETSPHINX_PLUGIN],[test "${enable_pocketsphinx_plugin}" = "yes" &&\
test "${found_pocketsphinx}" = "yes" &&\
test "${found_sphinxbase}" = "yes"])
#Enable Flite plugin
AC_ARG_ENABLE(flite-plugin,
[AC_HELP_STRING([--enable-flite-plugin ],[enable flite plugin])],
[enable_flite_plugin="$enableval"],
[enable_flite_plugin="no"])
if test "${enable_flite_plugin}" != "no"; then
UNIMRCP_CHECK_FLITE
fi
AM_CONDITIONAL([FLITE_PLUGIN],[test "${enable_flite_plugin}" = "yes" &&\
test "${found_flite}" = "yes"])
AM_CONDITIONAL(ISMAC, [test `uname -s` = Darwin])
AC_CONFIG_FILES([
@ -127,6 +180,8 @@ AC_CONFIG_FILES([
modules/mrcp-unirtsp/Makefile
plugins/Makefile
plugins/mrcp-cepstral/Makefile
plugins/mrcp-pocketsphinx/Makefile
plugins/mrcp-flite/Makefile
plugins/demo-synth/Makefile
plugins/demo-recog/Makefile
platforms/Makefile
@ -148,3 +203,16 @@ AC_CONFIG_FILES([
])
AC_OUTPUT
AC_MSG_NOTICE
AC_MSG_NOTICE([Report:])
AC_MSG_NOTICE([ UniMRCP: $UNI_DOTTED_VERSION])
AC_MSG_NOTICE([ APR: $apr_version])
AC_MSG_NOTICE([ APR-Util: $apu_version])
AC_MSG_NOTICE([ Sofia-SIP: $sofia_version])
AC_MSG_NOTICE([Plugins:])
AC_MSG_NOTICE([ Demo Synthesizer: $enable_demosynth_plugin])
AC_MSG_NOTICE([ Demo Recognizer: $enable_demorecog_plugin])
AC_MSG_NOTICE([ Cepstral: $enable_cepstral_plugin])
AC_MSG_NOTICE([ PocketSphinx: $enable_pocketsphinx_plugin])
AC_MSG_NOTICE([ Flite: $enable_flite_plugin])

@ -42,6 +42,9 @@ typedef enum {
/** Create activity detector */
MPF_DECLARE(mpf_activity_detector_t*) mpf_activity_detector_create(apr_pool_t *pool);
/** Reset activity detector */
MPF_DECLARE(void) mpf_activity_detector_reset(mpf_activity_detector_t *detector);
/** Set threshold of voice activity (silence) level */
MPF_DECLARE(void) mpf_activity_detector_level_set(mpf_activity_detector_t *detector, apr_size_t level_threshold);

@ -39,7 +39,9 @@ APT_BEGIN_EXTERN_C
typedef enum {
MPF_SAMPLE_RATE_NONE = 0x00,
MPF_SAMPLE_RATE_8000 = 0x01,
MPF_SAMPLE_RATE_16000 = 0x02
MPF_SAMPLE_RATE_16000 = 0x02,
MPF_SAMPLE_RATE_32000 = 0x04,
MPF_SAMPLE_RATE_48000 = 0x08
} mpf_sample_rates_e;
/** Codec descriptor declaration */

@ -53,6 +53,13 @@ MPF_DECLARE(mpf_activity_detector_t*) mpf_activity_detector_create(apr_pool_t *p
return detector;
}
/** Reset activity detector */
MPF_DECLARE(void) mpf_activity_detector_reset(mpf_activity_detector_t *detector)
{
detector->duration = 0;
detector->state = DETECTOR_STATE_INACTIVITY;
}
/** Set threshold of voice activity (silence) level */
MPF_DECLARE(void) mpf_activity_detector_level_set(mpf_activity_detector_t *detector, apr_size_t level_threshold)
{

@ -140,15 +140,17 @@ static const mpf_codec_descriptor_t g711a_descriptor = {
};
static const mpf_codec_attribs_t g711u_attribs = {
{G711u_CODEC_NAME, G711u_CODEC_NAME_LENGTH}, /* codec name */
8, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 /* sampling rates */
{G711u_CODEC_NAME, G711u_CODEC_NAME_LENGTH}, /* codec name */
8, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 |
MPF_SAMPLE_RATE_32000 | MPF_SAMPLE_RATE_48000 /* supported sampling rates */
};
static const mpf_codec_attribs_t g711a_attribs = {
{G711a_CODEC_NAME, G711a_CODEC_NAME_LENGTH}, /* codec name */
8, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 /* sampling rates */
{G711a_CODEC_NAME, G711a_CODEC_NAME_LENGTH}, /* codec name */
8, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 |
MPF_SAMPLE_RATE_32000 | MPF_SAMPLE_RATE_48000 /* supported sampling rates */
};
mpf_codec_t* mpf_codec_g711u_create(apr_pool_t *pool)

@ -84,15 +84,17 @@ static const mpf_codec_vtable_t l16_vtable = {
};
static const mpf_codec_attribs_t lpcm_attribs = {
{LPCM_CODEC_NAME, LPCM_CODEC_NAME_LENGTH}, /* codec name */
16, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 /* sampling rates */
{LPCM_CODEC_NAME, LPCM_CODEC_NAME_LENGTH}, /* codec name */
16, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 |
MPF_SAMPLE_RATE_32000 | MPF_SAMPLE_RATE_48000 /* supported sampling rates */
};
static const mpf_codec_attribs_t l16_attribs = {
{L16_CODEC_NAME, L16_CODEC_NAME_LENGTH}, /* codec name */
16, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 /* sampling rates */
{L16_CODEC_NAME, L16_CODEC_NAME_LENGTH}, /* codec name */
16, /* bits per sample */
MPF_SAMPLE_RATE_8000 | MPF_SAMPLE_RATE_16000 |
MPF_SAMPLE_RATE_32000 | MPF_SAMPLE_RATE_48000 /* supported sampling rates */
};
mpf_codec_descriptor_t* mpf_codec_lpcm_descriptor_create(apr_uint16_t sampling_rate, apr_byte_t channel_count, apr_pool_t *pool)

@ -74,6 +74,8 @@ struct mrcp_engine_channel_t {
mpf_termination_t *termination;
/** Back pointer to resource engine */
mrcp_resource_engine_t *engine;
/** Unique identifier (useful for traces) */
apt_str_t id;
/** Pool to allocate memory from */
apr_pool_t *pool;
};

@ -51,6 +51,7 @@ mrcp_engine_channel_t* mrcp_engine_channel_create(
channel->termination = termination;
channel->engine = engine;
channel->pool = pool;
apt_string_reset(&channel->id);
return channel;
}

@ -206,6 +206,7 @@ static mrcp_channel_t* mrcp_server_channel_create(mrcp_server_session_t *session
engine_channel = mrcp_server_engine_channel_create(session,resource_name);
if(engine_channel) {
engine_channel->id = session->base.id;
engine_channel->event_obj = channel;
engine_channel->event_vtable = &engine_channel_vtable;
channel->engine_channel = engine_channel;

@ -13,3 +13,11 @@ endif
if CEPSTRAL_PLUGIN
SUBDIRS += mrcp-cepstral
endif
if POCKETSPHINX_PLUGIN
SUBDIRS += mrcp-pocketsphinx
endif
if FLITE_PLUGIN
SUBDIRS += mrcp-flite
endif

@ -0,0 +1,19 @@
MAINTAINERCLEANFILES = Makefile.in
INCLUDES = -Iinclude \
-I$(top_srcdir)/libs/mrcp-engine/include \
-I$(top_srcdir)/libs/mrcp/include \
-I$(top_srcdir)/libs/mrcp/message/include \
-I$(top_srcdir)/libs/mrcp/control/include \
-I$(top_srcdir)/libs/mrcp/resources/include \
-I$(top_srcdir)/libs/mpf/include \
-I$(top_srcdir)/libs/apr-toolkit/include \
$(UNIMRCP_APR_INCLUDES) \
$(UNIMRCP_APU_INCLUDES) \
$(UNIMRCP_FLITE_INCLUDES)
plugin_LTLIBRARIES = mrcpflite.la
mrcpflite_la_SOURCES = src/mrcp_flite.c
mrcpflite_la_LDFLAGS = -module $(PLUGIN_LT_VERSION)
mrcpflite_la_LIBADD = $(UNIMRCP_FLITE_LIBS) -lm

@ -0,0 +1,165 @@
<?xml version="1.0" encoding="Windows-1252"?>
<VisualStudioProject
ProjectType="Visual C++"
Version="8.00"
Name="mrcpflite"
ProjectGUID="{56F6FB96-2BC7-4CAE-A8BF-6A0FAEC90556}"
RootNamespace="mrcpflite"
Keyword="Win32Proj"
>
<Platforms>
<Platform
Name="Win32"
/>
</Platforms>
<ToolFiles>
</ToolFiles>
<Configurations>
<Configuration
Name="Debug|Win32"
ConfigurationType="2"
InheritedPropertySheets="$(ProjectDir)..\..\build\vsprops\unidebug.vsprops;$(ProjectDir)..\..\build\vsprops\flite.vsprops;$(ProjectDir)..\..\build\vsprops\unimrcpplugin.vsprops"
CharacterSet="1"
>
<Tool
Name="VCPreBuildEventTool"
/>
<Tool
Name="VCCustomBuildTool"
/>
<Tool
Name="VCXMLDataGeneratorTool"
/>
<Tool
Name="VCWebServiceProxyGeneratorTool"
/>
<Tool
Name="VCMIDLTool"
/>
<Tool
Name="VCCLCompilerTool"
AdditionalIncludeDirectories="include"
/>
<Tool
Name="VCManagedResourceCompilerTool"
/>
<Tool
Name="VCResourceCompilerTool"
/>
<Tool
Name="VCPreLinkEventTool"
/>
<Tool
Name="VCLinkerTool"
/>
<Tool
Name="VCALinkTool"
/>
<Tool
Name="VCManifestTool"
/>
<Tool
Name="VCXDCMakeTool"
/>
<Tool
Name="VCBscMakeTool"
/>
<Tool
Name="VCFxCopTool"
/>
<Tool
Name="VCAppVerifierTool"
/>
<Tool
Name="VCWebDeploymentTool"
/>
<Tool
Name="VCPostBuildEventTool"
/>
</Configuration>
<Configuration
Name="Release|Win32"
ConfigurationType="2"
InheritedPropertySheets="$(ProjectDir)..\..\build\vsprops\unirelease.vsprops;$(ProjectDir)..\..\build\vsprops\flite.vsprops;$(ProjectDir)..\..\build\vsprops\unimrcpplugin.vsprops"
CharacterSet="1"
WholeProgramOptimization="1"
>
<Tool
Name="VCPreBuildEventTool"
/>
<Tool
Name="VCCustomBuildTool"
/>
<Tool
Name="VCXMLDataGeneratorTool"
/>
<Tool
Name="VCWebServiceProxyGeneratorTool"
/>
<Tool
Name="VCMIDLTool"
/>
<Tool
Name="VCCLCompilerTool"
AdditionalIncludeDirectories="include"
/>
<Tool
Name="VCManagedResourceCompilerTool"
/>
<Tool
Name="VCResourceCompilerTool"
/>
<Tool
Name="VCPreLinkEventTool"
/>
<Tool
Name="VCLinkerTool"
/>
<Tool
Name="VCALinkTool"
/>
<Tool
Name="VCManifestTool"
/>
<Tool
Name="VCXDCMakeTool"
/>
<Tool
Name="VCBscMakeTool"
/>
<Tool
Name="VCFxCopTool"
/>
<Tool
Name="VCAppVerifierTool"
/>
<Tool
Name="VCWebDeploymentTool"
/>
<Tool
Name="VCPostBuildEventTool"
/>
</Configuration>
</Configurations>
<References>
</References>
<Files>
<Filter
Name="include"
Filter="h;hpp;hxx;hm;inl;inc;xsd"
UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"
>
</Filter>
<Filter
Name="src"
Filter="cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx"
>
<File
RelativePath=".\src\mrcp_flite.c"
>
</File>
</Filter>
</Files>
<Globals>
</Globals>
</VisualStudioProject>

@ -0,0 +1,749 @@
/*
* Copyright 2008 Arsen Chaloyan
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*
* Some mandatory rules for plugin implementation.
* 1. Each plugin MUST contain the following function as an entry point of the plugin
* MRCP_PLUGIN_DECLARE(mrcp_resource_engine_t*) mrcp_plugin_create(apr_pool_t *pool)
* 2. One and only one response MUST be sent back to the received request.
* 3. Methods (callbacks) of the MRCP engine channel MUST not block.
* (asynch response can be sent from the context of other thread)
* 4. Methods (callbacks) of the MPF engine stream MUST not block.
*/
#include "mrcp_resource_engine.h"
#include "mrcp_synth_resource.h"
#include "mrcp_synth_header.h"
#include "mrcp_generic_header.h"
#include "mrcp_message.h"
#include "mpf_buffer.h"
#include "apr_time.h"
#include "apt_consumer_task.h"
#include "apt_log.h"
#include "flite.h"
typedef struct flite_synth_engine_t flite_synth_engine_t;
typedef struct flite_synth_channel_t flite_synth_channel_t;
typedef struct flite_synth_msg_t flite_synth_msg_t;
/** Declaration of synthesizer engine methods */
static apt_bool_t flite_synth_engine_destroy(mrcp_resource_engine_t *engine);
static apt_bool_t flite_synth_engine_open(mrcp_resource_engine_t *engine);
static apt_bool_t flite_synth_engine_close(mrcp_resource_engine_t *engine);
static mrcp_engine_channel_t* flite_synth_engine_channel_create(mrcp_resource_engine_t *engine, apr_pool_t *pool);
static const struct mrcp_engine_method_vtable_t engine_vtable = {
flite_synth_engine_destroy,
flite_synth_engine_open,
flite_synth_engine_close,
flite_synth_engine_channel_create
};
/** Declaration of synthesizer channel methods */
static apt_bool_t flite_synth_channel_destroy(mrcp_engine_channel_t *channel);
static apt_bool_t flite_synth_channel_open(mrcp_engine_channel_t *channel);
static apt_bool_t flite_synth_channel_close(mrcp_engine_channel_t *channel);
static apt_bool_t flite_synth_channel_request_process(mrcp_engine_channel_t *channel, mrcp_message_t *request);
static const struct mrcp_engine_channel_method_vtable_t channel_vtable = {
flite_synth_channel_destroy,
flite_synth_channel_open,
flite_synth_channel_close,
flite_synth_channel_request_process
};
/** Declaration of synthesizer audio stream methods */
static apt_bool_t flite_synth_stream_destroy(mpf_audio_stream_t *stream);
static apt_bool_t flite_synth_stream_open(mpf_audio_stream_t *stream);
static apt_bool_t flite_synth_stream_close(mpf_audio_stream_t *stream);
static apt_bool_t flite_synth_stream_read(mpf_audio_stream_t *stream, mpf_frame_t *frame);
static const mpf_audio_stream_vtable_t audio_stream_vtable = {
flite_synth_stream_destroy,
flite_synth_stream_open,
flite_synth_stream_close,
flite_synth_stream_read,
NULL,
NULL,
NULL
};
/** Declaration of flite synthesizer engine */
struct flite_synth_engine_t {
apt_consumer_task_t *task;
int iChannels;
apr_thread_mutex_t *guard;
};
// flite stuff
APT_BEGIN_EXTERN_C
cst_voice *register_cmu_us_awb(void);
void unregister_cmu_us_awb(cst_voice * v);
cst_voice *register_cmu_us_kal(void);
void unregister_cmu_us_kal(cst_voice * v);
cst_voice *register_cmu_us_rms(void);
void unregister_cmu_us_rms(cst_voice * v);
cst_voice *register_cmu_us_slt(void);
void unregister_cmu_us_slt(cst_voice * v);
APT_END_EXTERN_C
static struct {
cst_voice *awb;
cst_voice *kal;
cst_voice *rms;
cst_voice *slt;
} voices;
/** Declaration of flite synthesizer channel */
struct flite_synth_channel_t {
flite_synth_engine_t *flite_engine; // Back pointer to engine
mrcp_engine_channel_t *channel; // Engine channel base
mrcp_message_t *speak_request; // Active (in-progress) speak request
mrcp_message_t *stop_response; // Pending stop response
apt_bool_t paused; // Is paused
mpf_buffer_t *audio_buffer; // Audio buffer
int iId; // Synth channel simultaneous reference count
cst_voice *voice;
cst_wave *wave;
apr_pool_t *pool;
apt_consumer_task_t *task;
};
typedef enum flite_synth_msg_type_e {
flite_synth_MSG_OPEN_CHANNEL,
flite_synth_MSG_CLOSE_CHANNEL,
flite_synth_MSG_REQUEST_PROCESS
} flite_synth_msg_type_e;
/** Declaration of flite synthesizer task message */
struct flite_synth_msg_t {
flite_synth_msg_type_e type;
mrcp_engine_channel_t *channel;
mrcp_message_t *request;
};
/* mutex: may be flite library is not thread safe*/
static apr_thread_mutex_t *flite_mutex;
struct flite_speak_msg_t {
flite_synth_channel_t *channel;
mrcp_message_t *request;
};
typedef struct flite_speak_msg_t flite_speak_msg_t;
// all calls to the Flite API functions (after initialization)
// will be carried out using a separate task
static apt_bool_t flite_synth_msg_signal(flite_synth_msg_type_e type, mrcp_engine_channel_t *channel, mrcp_message_t *request);
static apt_bool_t flite_synth_msg_process(apt_task_t *task, apt_task_msg_t *msg);
static apt_bool_t flite_synth_channel_open_t(mrcp_engine_channel_t * channel);
static apt_bool_t flite_synth_channel_close_t(mrcp_engine_channel_t * channel);
// and we have a special task for the actual synthesis -
// the task is created when a mrcp speak message is received
static apt_bool_t flite_speak_msg_process(apt_task_t *task, apt_task_msg_t *msg);
/** Declare this macro to use log routine of the server where the plugin is loaded from */
MRCP_PLUGIN_LOGGER_IMPLEMENT
/** Create flite synthesizer engine */
MRCP_PLUGIN_DECLARE(mrcp_resource_engine_t*) mrcp_plugin_create(apr_pool_t *pool)
{
/* create flite engine */
flite_synth_engine_t *flite_engine = (flite_synth_engine_t *) apr_palloc(pool,sizeof(flite_synth_engine_t));
apt_task_msg_pool_t *msg_pool;
apt_task_vtable_t *task_vtable = 0;
flite_engine->iChannels = 0;
/* create task/thread to run flite engine in the context of this task */
msg_pool = apt_task_msg_pool_create_dynamic(sizeof(flite_synth_msg_t),pool);
flite_engine->task = apt_consumer_task_create(flite_engine,msg_pool,pool);
if (!flite_engine->task)
{
apt_log(APT_LOG_MARK, APT_PRIO_ERROR, "MRCP_PLUGIN_DECLARE cannot create task");
return NULL;
}
task_vtable = apt_consumer_task_vtable_get(flite_engine->task);
if (task_vtable)
{
task_vtable->process_msg = flite_synth_msg_process;
}
else
{
apt_log(APT_LOG_MARK, APT_PRIO_ERROR, "MRCP_PLUGIN_DECLARE cannot use task vtable");
return NULL;
}
/* create flite mutex */
if (apr_thread_mutex_create(&flite_mutex,APR_THREAD_MUTEX_DEFAULT,pool) != APR_SUCCESS)
{
apt_log(APT_LOG_MARK, APT_PRIO_ERROR,"Failed to create flite mutex");
return NULL;
}
/* create channel mutex */
if (apr_thread_mutex_create(&flite_engine->guard,APR_THREAD_MUTEX_DEFAULT,pool) != APR_SUCCESS)
{
apt_log(APT_LOG_MARK, APT_PRIO_ERROR, "Failed to create channel guard");
return NULL;
}
/* create resource engine base */
return mrcp_resource_engine_create(
MRCP_SYNTHESIZER_RESOURCE, /* MRCP resource identifier */
flite_engine, /* object to associate */
&engine_vtable, /* virtual methods table of resource engine */
pool); /* pool to allocate memory from */
}
/** Destroy synthesizer engine */
static apt_bool_t flite_synth_engine_destroy(mrcp_resource_engine_t *engine)
{
flite_synth_engine_t *flite_engine = (flite_synth_engine_t *) engine->obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_engine_destroy");
if(flite_engine->task) {
apt_task_t *task = apt_consumer_task_base_get(flite_engine->task);
apt_task_destroy(task);
flite_engine->task = NULL;
}
apr_thread_mutex_destroy(flite_engine->guard);
return TRUE;
}
/** Open synthesizer engine */
static apt_bool_t flite_synth_engine_open(mrcp_resource_engine_t *engine)
{
flite_synth_engine_t *flite_engine = (flite_synth_engine_t *) engine->obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_engine_open");
flite_init();
voices.awb = register_cmu_us_awb();
voices.kal = register_cmu_us_kal();
voices.rms = register_cmu_us_rms();
voices.slt = register_cmu_us_slt();
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "fliteInitialize success");
if (flite_engine->task) {
apt_task_t *task = apt_consumer_task_base_get(flite_engine->task);
apt_task_start(task);
}
return TRUE;
}
/** Close synthesizer engine */
static apt_bool_t flite_synth_engine_close(mrcp_resource_engine_t *engine)
{
flite_synth_engine_t *flite_engine = (flite_synth_engine_t *) engine->obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_engine_close");
if (flite_engine->task)
{
apt_task_t *task = apt_consumer_task_base_get(flite_engine->task);
apt_task_terminate(task,TRUE);
}
unregister_cmu_us_awb(voices.awb);
unregister_cmu_us_kal(voices.kal);
unregister_cmu_us_rms(voices.rms);
unregister_cmu_us_slt(voices.slt);
return TRUE;
}
/** Create flite synthesizer channel derived from engine channel base */
static mrcp_engine_channel_t* flite_synth_engine_channel_create(mrcp_resource_engine_t *engine, apr_pool_t *pool)
{
/* create flite synth channel */
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) apr_palloc(pool,sizeof(flite_synth_channel_t));
mpf_codec_descriptor_t *codec_descriptor = NULL;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_engine_channel_create");
// codec_descriptor = (mpf_codec_descriptor_t *) apr_palloc(pool,sizeof(mpf_codec_descriptor_t));
// mpf_codec_descriptor_init(codec_descriptor);
// codec_descriptor->channel_count = 1;
// codec_descriptor->payload_type = 96;
// apt_string_set(&codec_descriptor->name,"L16");
// codec_descriptor->sampling_rate = 16000;
//
synth_channel->flite_engine = (flite_synth_engine_t *) engine->obj;
synth_channel->speak_request = NULL; // no active speak request in progress
synth_channel->stop_response = NULL;
synth_channel->paused = FALSE;
synth_channel->pool = pool;
synth_channel->wave = NULL;
synth_channel->voice = NULL;
synth_channel->iId = 0;
/* create engine channel base */
synth_channel->channel = mrcp_engine_source_channel_create(
engine, /* resource engine */
&channel_vtable, /* virtual methods table of engine channel */
&audio_stream_vtable, /* virtual methods table of audio stream */
synth_channel, /* object to associate */
codec_descriptor, /* codec descriptor might be NULL by default */
pool); /* pool to allocate memory from */
if (!synth_channel->channel)
{
apt_log(APT_LOG_MARK, APT_PRIO_WARNING, "flite_synth_engine_channel_create failed");
return NULL;
}
synth_channel->audio_buffer = mpf_buffer_create(pool);
apr_thread_mutex_lock(synth_channel->flite_engine->guard);
synth_channel->iId = ++synth_channel->flite_engine->iChannels;
apr_thread_mutex_unlock(synth_channel->flite_engine->guard);
apt_log(APT_LOG_MARK, APT_PRIO_DEBUG, "flite_synth_engine_channel_create created channel %d", synth_channel->iId);
return synth_channel->channel;
}
/** Destroy engine channel */
static apt_bool_t flite_synth_channel_destroy(mrcp_engine_channel_t *channel)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_destroy - channel %d", synth_channel->iId);
if(synth_channel->task)
{
apt_task_t *task = apt_consumer_task_base_get(synth_channel->task);
if (!apt_task_destroy(task))
{
apt_log(APT_LOG_MARK, APT_PRIO_WARNING, "Speak task destroy failed - channel %d", synth_channel->iId);
}
else
{
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "Speak task destroyed - channel %d", synth_channel->iId);
}
}
synth_channel->task = NULL;
apr_thread_mutex_lock(synth_channel->flite_engine->guard);
synth_channel->flite_engine->iChannels--;
apr_thread_mutex_unlock(synth_channel->flite_engine->guard);
return TRUE;
}
/** Open engine channel (asynchronous response MUST be sent)*/
static apt_bool_t flite_synth_channel_open(mrcp_engine_channel_t *channel)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_open - channel %d", synth_channel->iId);
return flite_synth_msg_signal(flite_synth_MSG_OPEN_CHANNEL,channel,NULL);
}
/** Close engine channel (asynchronous response MUST be sent)*/
static apt_bool_t flite_synth_channel_close(mrcp_engine_channel_t *channel)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_close - channel %d", synth_channel->iId);
return flite_synth_msg_signal(flite_synth_MSG_CLOSE_CHANNEL,channel,NULL);
}
/** Process MRCP channel request (asynchronous response MUST be sent)*/
static apt_bool_t flite_synth_channel_request_process(mrcp_engine_channel_t *channel, mrcp_message_t *request)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_request_process - channel %d", synth_channel->iId);
return flite_synth_msg_signal(flite_synth_MSG_REQUEST_PROCESS,channel,request);
}
/** Open engine channel */
static apt_bool_t flite_synth_channel_open_t(mrcp_engine_channel_t *channel)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_open_t - channel %d", synth_channel->iId);
synth_channel->voice = voices.awb;
return TRUE;
}
static apt_bool_t flite_synth_channel_close_t(mrcp_engine_channel_t *channel)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_close_t - channel %d", synth_channel->iId);
if (synth_channel->task)
{
apt_task_t *task = apt_consumer_task_base_get(synth_channel->task);
if (!apt_task_terminate(task,TRUE))
{
apt_log(APT_LOG_MARK, APT_PRIO_WARNING, "Speak task terminate failed - channel %d", synth_channel->iId);
}
else
{
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "Speak task terminated - channel %d", synth_channel->iId);
apt_task_destroy(task);
synth_channel->task = 0;
}
}
if (synth_channel->wave)
{
delete_wave(synth_channel->wave);
synth_channel->wave = 0;
}
else
{
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "No wave buffer to delete - channel %d", synth_channel->iId);
}
return TRUE;
}
/** Process SPEAK request */
static apt_bool_t flite_synth_channel_speak(mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_speak - channel %d", synth_channel->iId);
if (!synth_channel->speak_request)
{
apt_task_msg_pool_t *msg_pool = apt_task_msg_pool_create_dynamic( sizeof(flite_speak_msg_t),synth_channel->pool);
apt_task_vtable_t *task_vtable = 0;
apt_task_t * task = 0;
apt_task_msg_t *msg = 0;
flite_speak_msg_t *flite_msg = 0;
/* create task/thread to run flite so this function is not blocking */
apt_log(APT_LOG_MARK,APT_PRIO_NOTICE,"Create flite speak task - channel: %d", synth_channel->iId);
synth_channel->task = apt_consumer_task_create(synth_channel, msg_pool, synth_channel->pool);
if (!synth_channel->task)
{
apt_log(APT_LOG_MARK,APT_PRIO_ERROR, "flite_synth_channel_speak failed to create flite speak task - channel:%d", synth_channel->iId);
return FALSE;
}
task_vtable = apt_consumer_task_vtable_get(synth_channel->task);
if (!task_vtable)
{
apt_log(APT_LOG_MARK,APT_PRIO_ERROR, "flite_synth_channel_speak cannot use flite speak task vtable - channel:%d", synth_channel->iId);
return FALSE;
}
task_vtable->process_msg = flite_speak_msg_process;
synth_channel->speak_request = request;
apt_log(APT_LOG_MARK,APT_PRIO_INFO, "Start task - channel %d", synth_channel->iId);
task = apt_consumer_task_base_get(synth_channel->task);
if (apt_task_start(task) == FALSE)
{
apt_log(APT_LOG_MARK,APT_PRIO_ERROR, "flite_synth_channel_speak failed to start task - channel: %d", synth_channel->iId);
apt_task_destroy(task);
return FALSE;
}
msg = apt_task_msg_acquire(msg_pool);
msg->type = TASK_MSG_USER;
flite_msg = (flite_speak_msg_t*) msg->data;
flite_msg->channel = synth_channel;
flite_msg->request = request;
apt_log(APT_LOG_MARK,APT_PRIO_INFO, "Send signal to start speech synthesis - channel:%d", synth_channel->iId);
if (apt_task_msg_signal(task,msg))
{
response->start_line.request_state = MRCP_REQUEST_STATE_INPROGRESS;
mrcp_engine_channel_message_send(channel,response);
}
else
{
apt_log(APT_LOG_MARK,APT_PRIO_INFO, "Failed to send signal to start speech synthesis - channel:%d", synth_channel->iId);
apt_task_destroy(task);
return FALSE;
}
}
return TRUE;
}
static apt_bool_t flite_speak_msg_process(apt_task_t *task, apt_task_msg_t *msg)
{
flite_speak_msg_t *flite_msg = (flite_speak_msg_t*)msg->data;
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) flite_msg->channel;
apr_time_t start = 0;
apr_time_t elapsed = 0;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "< flite_speak_msg_process speak - channel %d", synth_channel->iId);
// probably not necessary !?
mpf_buffer_restart(synth_channel->audio_buffer);
// just sequential stuff
start = apr_time_now(); // in microsec
if (synth_channel->speak_request->body.length)
{
cst_wave * wave = synth_channel->wave;
wave = flite_text_to_wave(synth_channel->speak_request->body.buf, synth_channel->voice);
}
elapsed = (apr_time_now() - start)/1000;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "TTS (chan %d) took %ld millisec", synth_channel->iId, elapsed);
if (!synth_channel->stop_response)
{
// this will notify the callback that feeds the client that synthesis is complete
mpf_buffer_event_write(synth_channel->audio_buffer, MEDIA_FRAME_TYPE_EVENT);
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "> flite_speak_msg_process speak - end of TTS - %d", synth_channel->iId);
}
else
{
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "> flite_speak_msg_process speak - channel %d", synth_channel->iId);
}
return TRUE;
}
/** Process STOP request */
static apt_bool_t flite_synth_channel_stop(mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_stop - channel %d", synth_channel->iId);
/* store the request, make sure there is no more activity and only then send the response */
/* TODO this should probably be mutexed */
synth_channel->stop_response = response;
return TRUE;
}
/** Process PAUSE request */
static apt_bool_t flite_synth_channel_pause(mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_pause - channel %d", synth_channel->iId);
synth_channel->paused = TRUE;
/* send asynchronous response */
mrcp_engine_channel_message_send(channel,response);
return TRUE;
}
/** Process RESUME request */
static apt_bool_t flite_synth_channel_resume(mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_resume - channel %d", synth_channel->iId);
synth_channel->paused = FALSE;
/* send asynchronous response */
mrcp_engine_channel_message_send(channel,response);
return TRUE;
}
/** Dispatch MRCP request */
static apt_bool_t flite_synth_channel_request_dispatch(mrcp_engine_channel_t *channel, mrcp_message_t *request)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_bool_t processed = FALSE;
mrcp_message_t *response = mrcp_response_create(request,request->pool);
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_channel_request_dispatch - channel %d", synth_channel->iId);
switch(request->start_line.method_id) {
case SYNTHESIZER_SET_PARAMS:
// TODO set voices
// if (!strcasecmp(voice_name, "awb")) {
// synth_channel->voice = voices.awb;
// } else if (!strcasecmp(voice_name, "kal")) {
// synth_channel->voice = voices.awb;
// } else if (!strcasecmp(voice_name, "rms")) {
// synth_channel->voice = voices.awb;
// } else if (!strcasecmp(voice_name, "slt")) {
// synth_channel->voice = voices.awb;
// } else {
// apt_log(APT_LOG_MARK, APT_PRIO_INFO, "Valid voice names are awb, kal, rms or slt");
// }
break;
case SYNTHESIZER_GET_PARAMS:
break;
case SYNTHESIZER_SPEAK:
processed = flite_synth_channel_speak(channel,request,response);
break;
case SYNTHESIZER_STOP:
processed = flite_synth_channel_stop(channel,request,response);
break;
case SYNTHESIZER_PAUSE:
processed = flite_synth_channel_pause(channel,request,response);
break;
case SYNTHESIZER_RESUME:
processed = flite_synth_channel_resume(channel,request,response);
break;
case SYNTHESIZER_BARGE_IN_OCCURRED:
processed = flite_synth_channel_stop(channel,request,response);
break;
case SYNTHESIZER_CONTROL:
break;
case SYNTHESIZER_DEFINE_LEXICON:
break;
default:
break;
}
if(processed == FALSE) {
/* send asynchronous response for not handled request */
mrcp_engine_channel_message_send(channel,response);
}
return TRUE;
}
/** Callback is called from MPF engine context to destroy any additional data associated with audio stream */
static apt_bool_t flite_synth_stream_destroy(mpf_audio_stream_t *stream)
{
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_stream_destroy");
return TRUE;
}
/** Callback is called from MPF engine context to perform any action before open */
static apt_bool_t flite_synth_stream_open(mpf_audio_stream_t *stream)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) stream->obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_stream_open - channel %d", synth_channel->iId);
return TRUE;
}
/** Callback is called from MPF engine context to perform any action after close */
static apt_bool_t flite_synth_stream_close(mpf_audio_stream_t *stream)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) stream->obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_stream_close - channel %d", synth_channel->iId);
return TRUE;
}
/** Raise SPEAK-COMPLETE event */
static apt_bool_t flite_synth_speak_complete_raise(flite_synth_channel_t *synth_channel)
{
mrcp_message_t *message = 0;
mrcp_synth_header_t * synth_header = 0;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_speak_complete_raise - channel %d", synth_channel->iId);
if (!synth_channel->speak_request) {
return FALSE;
}
message = mrcp_event_create(
synth_channel->speak_request,
SYNTHESIZER_SPEAK_COMPLETE,
synth_channel->speak_request->pool);
if (!message) {
return FALSE;
}
/* get/allocate synthesizer header */
synth_header = (mrcp_synth_header_t *) mrcp_resource_header_prepare(message);
if (synth_header) {
/* set completion cause */
synth_header->completion_cause = SYNTHESIZER_COMPLETION_CAUSE_NORMAL;
mrcp_resource_header_property_add(message,SYNTHESIZER_HEADER_COMPLETION_CAUSE);
}
/* set request state */
message->start_line.request_state = MRCP_REQUEST_STATE_COMPLETE;
synth_channel->speak_request = NULL;
/* send asynch event */
return mrcp_engine_channel_message_send(synth_channel->channel,message);
}
/** Callback is called from MPF engine context to read/get new frame */
static apt_bool_t flite_synth_stream_read(mpf_audio_stream_t *stream, mpf_frame_t *frame)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) stream->obj;
cst_wave * wave = synth_channel->wave;
if (synth_channel->stop_response && synth_channel->speak_request)
{
/* send asynchronous response to STOP request */
mrcp_engine_channel_message_send(synth_channel->channel, synth_channel->stop_response);
synth_channel->speak_request = NULL;
synth_channel->paused = FALSE;
return TRUE;
}
if (wave && cst_wave_num_samples(wave))
{
mpf_buffer_audio_write(synth_channel->audio_buffer, cst_wave_samples(wave), cst_wave_num_samples(wave) * 2);
}
/* check if there is active SPEAK request and it isn't in paused state */
if (synth_channel->speak_request && synth_channel->paused == FALSE)
{
/* normal processing */
mpf_buffer_frame_read(synth_channel->audio_buffer,frame);
// apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_stream_read - channel %d - size %d", synth_channel->iId, mpf_buffer_get_size(synth_channel->audio_buffer));
if((frame->type & MEDIA_FRAME_TYPE_EVENT) == MEDIA_FRAME_TYPE_EVENT)
{
flite_synth_speak_complete_raise(synth_channel);
}
}
return TRUE;
}
static apt_bool_t flite_synth_msg_signal(flite_synth_msg_type_e type, mrcp_engine_channel_t *channel, mrcp_message_t *request)
{
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) channel->method_obj;
apt_bool_t status = FALSE;
flite_synth_engine_t *flite_engine = synth_channel->flite_engine;
apt_task_t *task = apt_consumer_task_base_get(flite_engine->task);
apt_task_msg_t *msg = apt_task_msg_get(task);
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_msg_signal - channel %d", synth_channel->iId);
if (msg) {
flite_synth_msg_t *flite_msg;
msg->type = TASK_MSG_USER;
flite_msg = (flite_synth_msg_t*) msg->data;
flite_msg->type = type;
flite_msg->channel = channel;
flite_msg->request = request;
status = apt_task_msg_signal(task,msg);
}
else
{
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_msg_signal - task msg not found %d", synth_channel->iId);
}
return status;
}
static apt_bool_t flite_synth_msg_process(apt_task_t *task, apt_task_msg_t *msg)
{
flite_synth_msg_t *flite_msg = (flite_synth_msg_t*)msg->data;
flite_synth_channel_t *synth_channel = (flite_synth_channel_t *) flite_msg->channel->method_obj;
apt_log(APT_LOG_MARK, APT_PRIO_INFO, "flite_synth_msg_process - channel %d", synth_channel->iId);
switch(flite_msg->type) {
case flite_synth_MSG_OPEN_CHANNEL:
/* open channel and send asynch response */
flite_synth_channel_open_t(flite_msg->channel);
mrcp_engine_channel_open_respond(flite_msg->channel,TRUE);
break;
case flite_synth_MSG_CLOSE_CHANNEL:
/* close channel, make sure there is no activity and send asynch response */
flite_synth_channel_close_t(flite_msg->channel);
mrcp_engine_channel_close_respond(flite_msg->channel);
break;
case flite_synth_MSG_REQUEST_PROCESS:
flite_synth_channel_request_dispatch(flite_msg->channel,flite_msg->request);
break;
default:
break;
}
return TRUE;
}

@ -8,9 +8,13 @@ INCLUDES = -Iinclude \
-I$(top_srcdir)/libs/mrcp/resources/include \
-I$(top_srcdir)/libs/mpf/include \
-I$(top_srcdir)/libs/apr-toolkit/include \
$(UNIMRCP_APR_INCLUDES) $(UNIMRCP_APU_INCLUDES)
$(UNIMRCP_APR_INCLUDES) \
$(UNIMRCP_APU_INCLUDES) \
$(UNIMRCP_SPHINXBASE_INCLUDES) \
$(UNIMRCP_POCKETSPHINX_INCLUDES)
plugin_LTLIBRARIES = mrcppocketsphinx.la
mrcppocketsphinx_la_SOURCES = src/mrcp_pocketsphinx.c.c
mrcppocketsphinx_la_SOURCES = src/mrcp_pocketsphinx.c
mrcppocketsphinx_la_LDFLAGS = -module $(PLUGIN_LT_VERSION)
mrcppocketsphinx_la_LIBADD = $(UNIMRCP_POCKETSPHINX_LIBS) $(UNIMRCP_SPHINXBASE_LIBS) -lm

@ -24,6 +24,11 @@
* 4. Methods (callbacks) of the MPF engine stream MUST not block.
*/
#include <pocketsphinx.h>
#include <apr_thread_cond.h>
#include <apr_thread_proc.h>
#include <apr_tables.h>
#include <apr_file_io.h>
#include "mrcp_resource_engine.h"
#include "mrcp_recog_resource.h"
#include "mrcp_recog_header.h"
@ -33,24 +38,27 @@
#include "apt_log.h"
#define RECOGNIZER_SIDRES(recognizer) (recognizer)->channel->id.buf, "pocketsphinx"
typedef struct pocketsphinx_engine_t pocketsphinx_engine_t;
typedef struct pocketsphinx_recognizer_t pocketsphinx_recognizer_t;
typedef struct pocketsphinx_properties_t pocketsphinx_properties_t;
/** Declaration of recognizer engine methods */
/** Methods of recognition engine */
static apt_bool_t pocketsphinx_engine_destroy(mrcp_resource_engine_t *engine);
static apt_bool_t pocketsphinx_engine_open(mrcp_resource_engine_t *engine);
static apt_bool_t pocketsphinx_engine_close(mrcp_resource_engine_t *engine);
static mrcp_engine_channel_t* pocketsphinx_recognizer_create(mrcp_resource_engine_t *engine, apr_pool_t *pool);
static mrcp_engine_channel_t* pocketsphinx_engine_recognizer_create(mrcp_resource_engine_t *engine, apr_pool_t *pool);
static const struct mrcp_engine_method_vtable_t engine_vtable = {
pocketsphinx_engine_destroy,
pocketsphinx_engine_open,
pocketsphinx_engine_close,
pocketsphinx_recognizer_create
pocketsphinx_engine_recognizer_create
};
/** Declaration of recognizer channel methods */
/** Methods of recognition channel (recognizer) */
static apt_bool_t pocketsphinx_recognizer_destroy(mrcp_engine_channel_t *channel);
static apt_bool_t pocketsphinx_recognizer_open(mrcp_engine_channel_t *channel);
static apt_bool_t pocketsphinx_recognizer_close(mrcp_engine_channel_t *channel);
@ -63,43 +71,94 @@ static const struct mrcp_engine_channel_method_vtable_t channel_vtable = {
pocketsphinx_recognizer_request_process
};
/** Declaration of recognizer audio stream methods */
static apt_bool_t pocketsphinx_stream_destroy(mpf_audio_stream_t *stream);
static apt_bool_t pocketsphinx_stream_open(mpf_audio_stream_t *stream);
static apt_bool_t pocketsphinx_stream_close(mpf_audio_stream_t *stream);
/** Methods of audio stream to recognize */
static apt_bool_t pocketsphinx_stream_write(mpf_audio_stream_t *stream, const mpf_frame_t *frame);
static const mpf_audio_stream_vtable_t audio_stream_vtable = {
pocketsphinx_stream_destroy,
NULL,
NULL,
NULL,
pocketsphinx_stream_open,
pocketsphinx_stream_close,
NULL, /* destroy */
NULL, /* open_rx */
NULL, /* close_rx */
NULL, /* read_frame */
NULL, /* open_tx */
NULL, /* close_tx */
pocketsphinx_stream_write
};
/** Declaration of pocketsphinx engine */
/** Pocketsphinx engine (engine is an aggregation of recognizers) */
struct pocketsphinx_engine_t {
mrcp_resource_engine_t *base;
};
/** Declaration of pocketsphinx recognizer */
/** Pocketsphinx properties */
struct pocketsphinx_properties_t {
const char *dictionary;
const char *model_8k;
const char *model_16k;
apr_size_t noinput_timeout;
apr_size_t recognition_timeout;
apr_size_t partial_result_timeout;
};
/** Pocketsphinx channel (recognizer) */
struct pocketsphinx_recognizer_t {
/** Back pointer to engine */
pocketsphinx_engine_t *engine;
pocketsphinx_engine_t *engine;
/** Engine channel base */
mrcp_engine_channel_t *channel;
mrcp_engine_channel_t *channel;
/** Actual recognizer object */
ps_decoder_t *decoder;
/** Configuration */
cmd_ln_t *config;
/** Properties (to be loaded from config file) */
pocketsphinx_properties_t properties;
/** Is input timer started */
apt_bool_t is_input_timer_on;
/** Noinput timeout */
apr_size_t noinput_timeout;
/** Recognition timeout */
apr_size_t recognition_timeout;
/** Timeout elapsed since the last partial result checking */
apr_size_t partial_result_timeout;
/** Last (partially) recognized result */
const char *last_result;
/** Active grammar identifier (content-id) */
const char *grammar_id;
/** Table of defined grammars (key=content-id, value=grammar-file-path) */
apr_table_t *grammar_table;
/** Voice activity detector */
mpf_activity_detector_t *detector;
/** Thread to run recognition in */
apr_thread_t *thread;
/** Conditional wait object */
apr_thread_cond_t *wait_object;
/** Mutex of the wait object */
apr_thread_mutex_t *mutex;
/** Pending request from client stack to recognizer */
mrcp_message_t *request;
/** Pending event from mpf layer to recognizer */
mrcp_message_t *complete_event;
/** In-progress RECOGNIZE request */
mrcp_message_t *inprogress_recog;
/** Pending STOP response */
mrcp_message_t *stop_response;
/** Is recognition channel being closed */
apt_bool_t close_requested;
};
/** Declare this macro to use log routine of the server, plugin is loaded from */
MRCP_PLUGIN_LOGGER_IMPLEMENT
static void* APR_THREAD_FUNC pocketsphinx_recognizer_run(apr_thread_t *thread, void *data);
/** Create pocketsphinx engine (engine is an aggregation of recognizers) */
MRCP_PLUGIN_DECLARE(mrcp_resource_engine_t*) mrcp_plugin_create(apr_pool_t *pool)
{
pocketsphinx_engine_t *engine = apr_palloc(pool,sizeof(pocketsphinx_engine_t));
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Create PocketSphinx Engine");
/* create resource engine base */
engine->base = mrcp_resource_engine_create(
@ -110,27 +169,49 @@ MRCP_PLUGIN_DECLARE(mrcp_resource_engine_t*) mrcp_plugin_create(apr_pool_t *pool
return engine->base;
}
/** Destroy pocketsphinx engine */
static apt_bool_t pocketsphinx_engine_destroy(mrcp_resource_engine_t *engine)
{
return TRUE;
}
/** Open pocketsphinx engine */
static apt_bool_t pocketsphinx_engine_open(mrcp_resource_engine_t *engine)
{
return TRUE;
}
/** Close pocketsphinx engine */
static apt_bool_t pocketsphinx_engine_close(mrcp_resource_engine_t *engine)
{
return TRUE;
}
/** Create pocketsphinx recognizer */
static mrcp_engine_channel_t* pocketsphinx_recognizer_create(mrcp_resource_engine_t *engine, apr_pool_t *pool)
static mrcp_engine_channel_t* pocketsphinx_engine_recognizer_create(mrcp_resource_engine_t *engine, apr_pool_t *pool)
{
mrcp_engine_channel_t *channel;
pocketsphinx_recognizer_t *recognizer = apr_palloc(pool,sizeof(pocketsphinx_recognizer_t));
// recognizer->engine = engine;
recognizer->decoder = NULL;
recognizer->config = NULL;
recognizer->is_input_timer_on = FALSE;
recognizer->noinput_timeout = 0;
recognizer->recognition_timeout = 0;
recognizer->partial_result_timeout = 0;
recognizer->last_result = NULL;
recognizer->detector = NULL;
recognizer->thread = NULL;
recognizer->wait_object = NULL;
recognizer->mutex = NULL;
recognizer->request = NULL;
recognizer->complete_event = NULL;
recognizer->inprogress_recog = FALSE;
recognizer->stop_response = NULL;
recognizer->close_requested = FALSE;
recognizer->grammar_id = NULL;
recognizer->grammar_table = apr_table_make(pool,1);
/* create engine channel base */
channel = mrcp_engine_sink_channel_create(
engine, /* resource engine */
@ -144,55 +225,452 @@ static mrcp_engine_channel_t* pocketsphinx_recognizer_create(mrcp_resource_engin
return channel;
}
/** Destroy engine channel */
/** Destroy pocketsphinx recognizer */
static apt_bool_t pocketsphinx_recognizer_destroy(mrcp_engine_channel_t *channel)
{
return TRUE;
}
/** Open engine channel (asynchronous response MUST be sent)*/
/** Open pocketsphinx recognizer (asynchronous response MUST be sent) */
static apt_bool_t pocketsphinx_recognizer_open(mrcp_engine_channel_t *channel)
{
return mrcp_engine_channel_open_respond(channel,TRUE);
apr_status_t rv;
pocketsphinx_recognizer_t *recognizer = channel->method_obj;
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Open Channel "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
apr_thread_mutex_create(&recognizer->mutex,APR_THREAD_MUTEX_DEFAULT,channel->pool);
apr_thread_cond_create(&recognizer->wait_object,channel->pool);
/* Launch a thread to run recognition in */
rv = apr_thread_create(&recognizer->thread,NULL,pocketsphinx_recognizer_run,recognizer,channel->pool);
if(rv != APR_SUCCESS) {
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Failed to Launch Thread "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
apr_thread_mutex_destroy(recognizer->mutex);
recognizer->mutex = NULL;
apr_thread_cond_destroy(recognizer->wait_object);
recognizer->wait_object = NULL;
return mrcp_engine_channel_open_respond(channel,FALSE);
}
return TRUE;
}
/** Close engine channel (asynchronous response MUST be sent)*/
/** Close pocketsphinx recognizer (asynchronous response MUST be sent)*/
static apt_bool_t pocketsphinx_recognizer_close(mrcp_engine_channel_t *channel)
{
pocketsphinx_recognizer_t *recognizer = channel->method_obj;
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Close Channel "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
if(recognizer->thread) {
apr_status_t rv;
/* Signal recognition thread to terminate */
apr_thread_mutex_lock(recognizer->mutex);
recognizer->close_requested = TRUE;
apr_thread_cond_signal(recognizer->wait_object);
apr_thread_mutex_unlock(recognizer->mutex);
apr_thread_join(&rv,recognizer->thread);
recognizer->thread = NULL;
apr_thread_mutex_destroy(recognizer->mutex);
recognizer->mutex = NULL;
apr_thread_cond_destroy(recognizer->wait_object);
recognizer->wait_object = NULL;
}
return mrcp_engine_channel_close_respond(channel);
}
/** Process RECOGNIZE request */
static apt_bool_t pocketsphinx_recognize(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request, mrcp_message_t *response)
{
return TRUE;
}
/** Process STOP request */
static apt_bool_t pocketsphinx_stop(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request, mrcp_message_t *response)
{
return TRUE;
}
/** Process MRCP request (asynchronous response MUST be sent)*/
static apt_bool_t pocketsphinx_recognizer_request_process(mrcp_engine_channel_t *channel, mrcp_message_t *request)
{
pocketsphinx_recognizer_t *recognizer = channel->method_obj;
/* Store request and signal recognition thread to process the request */
apr_thread_mutex_lock(recognizer->mutex);
recognizer->request = request;
apr_thread_cond_signal(recognizer->wait_object);
apr_thread_mutex_unlock(recognizer->mutex);
return TRUE;
}
/** Load pocketsphinx properties [RECOG] */
static apt_bool_t pocketsphinx_properties_load(pocketsphinx_recognizer_t *recognizer)
{
mrcp_engine_channel_t *channel = recognizer->channel;
const apt_dir_layout_t *dir_layout = channel->engine->dir_layout;
pocketsphinx_properties_t *properties = &recognizer->properties;
properties->dictionary = apt_datadir_filepath_get(dir_layout,"pocketsphinx/default.dic",channel->pool);
properties->model_8k = apt_datadir_filepath_get(dir_layout,"pocketsphinx/communicator",channel->pool);
properties->model_16k = apt_datadir_filepath_get(dir_layout,"pocketsphinx/wsj1",channel->pool);
properties->noinput_timeout = 5000;
properties->recognition_timeout = 15000;
properties->partial_result_timeout = 100;
return TRUE;
}
/** Initialize pocketsphinx decoder [RECOG] */
static apt_bool_t pocketsphinx_decoder_init(pocketsphinx_recognizer_t *recognizer, const char *grammar)
{
apt_log(APT_LOG_MARK,APT_PRIO_DEBUG,"Init Config "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
recognizer->config = cmd_ln_init(recognizer->config, ps_args(), FALSE,
"-samprate", "8000",
"-hmm", recognizer->properties.model_8k,
"-jsgf", grammar,
"-dict", recognizer->properties.dictionary,
"-frate", "50",
"-silprob", "0.005",
NULL);
if(!recognizer->config) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Failed to Init Config "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
return FALSE;
}
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Init Decoder "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
if(recognizer->decoder) {
if(ps_reinit(recognizer->decoder,recognizer->config) < 0) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Failed to Reinit Decoder "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
return FALSE;
}
}
else {
recognizer->decoder = ps_init(recognizer->config);
if(!recognizer->decoder) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Failed to Init Decoder "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
return FALSE;
}
}
if(!recognizer->detector) {
recognizer->detector = mpf_activity_detector_create(recognizer->channel->pool);
mpf_activity_detector_level_set(recognizer->detector,50);
}
return TRUE;
}
/** Build pocketsphinx recognized result [RECOG] */
static apt_bool_t pocketsphinx_result_build(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *message)
{
apt_str_t *body = &message->body;
if(!recognizer->last_result || !recognizer->grammar_id) {
return FALSE;
}
body->buf = apr_psprintf(message->pool,
"<?xml version=\"1.0\"?>\n"
"<result grammar=\"%s\">\n"
" <interpretation grammar=\"%s\" confidence=\"%d\">\n"
" <input mode=\"speech\">%s</input>\n"
" </interpretation>\n"
"</result>\n",
recognizer->grammar_id,
recognizer->grammar_id,
99,
recognizer->last_result,
recognizer->last_result);
if(body->buf) {
mrcp_generic_header_t *generic_header;
generic_header = mrcp_generic_header_prepare(message);
if(generic_header) {
/* set content type */
apt_string_assign(&generic_header->content_type,"application/x-nlsml",message->pool);
mrcp_generic_header_property_add(message,GENERIC_HEADER_CONTENT_TYPE);
}
body->length = strlen(body->buf);
}
return TRUE;
}
/** Clear pocketsphinx grammars [RECOG] */
static apt_bool_t pocketsphinx_grammars_clear(pocketsphinx_recognizer_t *recognizer)
{
const apr_array_header_t *tarr = apr_table_elts(recognizer->grammar_table);
const apr_table_entry_t *telts = (const apr_table_entry_t*)tarr->elts;
int i;
for(i = 0; i < tarr->nelts; i++) {
const char *grammar_file_path = telts[i].val;
if(grammar_file_path) {
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Remove Grammar File [%s] "APT_SIDRES_FMT,
grammar_file_path,RECOGNIZER_SIDRES(recognizer));
apr_file_remove(grammar_file_path,recognizer->channel->pool);
}
}
apr_table_clear(recognizer->grammar_table);
return TRUE;
}
/** Process DEFINE-GRAMMAR request [RECOG] */
static apt_bool_t pocketsphinx_define_grammar(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request, mrcp_message_t *response)
{
const char *content_type = NULL;
const char *content_id = NULL;
apt_str_t *grammar = NULL;
mrcp_engine_channel_t *channel = recognizer->channel;
/* get recognizer header */
mrcp_generic_header_t *generic_header = mrcp_generic_header_get(request);
mrcp_recog_header_t *recog_header = mrcp_resource_header_prepare(response);
if(!generic_header || !recog_header) {
return FALSE;
}
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_SUCCESS;
mrcp_resource_header_property_add(response,RECOGNIZER_HEADER_COMPLETION_CAUSE);
/* content-id must be specified */
if(mrcp_generic_header_property_check(request,GENERIC_HEADER_CONTENT_ID) == TRUE) {
content_id = generic_header->content_id.buf;
}
if(!content_id) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Missing Content-Id "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
response->start_line.status_code = MRCP_STATUS_CODE_MISSING_PARAM;
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_GRAM_LOAD_FAILURE;
return FALSE;
}
if(mrcp_generic_header_property_check(request,GENERIC_HEADER_CONTENT_LENGTH) == TRUE) {
grammar = &request->body;
}
if(grammar) {
/* load grammar */
const apt_dir_layout_t *dir_layout = channel->engine->dir_layout;
const char *grammar_file_path = NULL;
const char *grammar_file_name = NULL;
apr_file_t *fd = NULL;
apr_status_t rv;
apr_size_t size;
/* content-type must be specified */
if(mrcp_generic_header_property_check(request,GENERIC_HEADER_CONTENT_TYPE) == TRUE) {
content_type = generic_header->content_type.buf;
}
if(!content_type) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Missing Content-Type "APT_SIDRES_FMT,RECOGNIZER_SIDRES(recognizer));
response->start_line.status_code = MRCP_STATUS_CODE_MISSING_PARAM;
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_GRAM_LOAD_FAILURE;
return FALSE;
}
/* only JSGF grammar is supported */
if(strstr(content_type,"jsgf") == NULL) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Not Supported Content-Type [%s] "APT_SIDRES_FMT,
content_type,RECOGNIZER_SIDRES(recognizer));
response->start_line.status_code = MRCP_STATUS_CODE_UNSUPPORTED_PARAM_VALUE;
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_GRAM_LOAD_FAILURE;
return FALSE;
}
grammar_file_name = apr_psprintf(channel->pool,"pocketsphinx/%s-%s.gram",channel->id.buf,content_id);
grammar_file_path = apt_datadir_filepath_get(dir_layout,grammar_file_name,channel->pool);
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Create Grammar File [%s] "APT_SIDRES_FMT,
grammar_file_path,RECOGNIZER_SIDRES(recognizer));
rv = apr_file_open(&fd,grammar_file_path,APR_CREATE|APR_TRUNCATE|APR_WRITE|APR_BINARY,
APR_OS_DEFAULT,channel->pool);
if(rv != APR_SUCCESS) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Cannot Open Grammar File to Write [%s] "APT_SIDRES_FMT,
grammar_file_path,RECOGNIZER_SIDRES(recognizer));
response->start_line.status_code = MRCP_STATUS_CODE_METHOD_FAILED;
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_GRAM_LOAD_FAILURE;
return FALSE;
}
size = grammar->length;
apr_file_write(fd,grammar->buf,&size);
apr_file_close(fd);
/* init pocketsphinx decoder */
if(pocketsphinx_decoder_init(recognizer,grammar_file_path) == TRUE) {
recognizer->grammar_id = content_id;
apr_table_setn(recognizer->grammar_table,content_id,grammar_file_path);
}
else {
response->start_line.status_code = MRCP_STATUS_CODE_METHOD_FAILED;
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_GRAM_LOAD_FAILURE;
apr_file_remove(grammar_file_path,channel->pool);
return FALSE;
}
}
else {
/* unload grammar */
const char *grammar_file_path = apr_table_get(recognizer->grammar_table,content_id);
if(grammar_file_path) {
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Remove Grammar File [%s] "APT_SIDRES_FMT,
grammar_file_path,RECOGNIZER_SIDRES(recognizer));
apr_file_remove(grammar_file_path,channel->pool);
apr_table_unset(recognizer->grammar_table,content_id);
}
}
/* send asynchronous response */
mrcp_engine_channel_message_send(channel,response);
return TRUE;
}
/** Process RECOGNIZE request [RECOG] */
static apt_bool_t pocketsphinx_recognize(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request, mrcp_message_t *response)
{
mrcp_recog_header_t *request_recog_header;
mrcp_recog_header_t *response_recog_header = mrcp_resource_header_prepare(response);
if(!response_recog_header) {
return FALSE;
}
response_recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_SUCCESS;
mrcp_resource_header_property_add(response,RECOGNIZER_HEADER_COMPLETION_CAUSE);
if(!recognizer->decoder || ps_start_utt(recognizer->decoder, NULL) < 0) {
response->start_line.status_code = MRCP_STATUS_CODE_METHOD_FAILED;
response_recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_ERROR;
return FALSE;
}
recognizer->is_input_timer_on = TRUE;
/* get recognizer header */
request_recog_header = mrcp_resource_header_get(request);
if(request_recog_header) {
if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_START_INPUT_TIMERS) == TRUE) {
recognizer->is_input_timer_on = request_recog_header->start_input_timers;
}
}
response->start_line.request_state = MRCP_REQUEST_STATE_INPROGRESS;
/* send asynchronous response */
mrcp_engine_channel_message_send(recognizer->channel,response);
/* reset */
mpf_activity_detector_reset(recognizer->detector);
recognizer->noinput_timeout = 0;
recognizer->recognition_timeout = 0;
recognizer->partial_result_timeout = 0;
recognizer->last_result = NULL;
recognizer->complete_event = NULL;
recognizer->inprogress_recog = request;
return TRUE;
}
/** Process GET-RESULTS request [RECOG] */
static apt_bool_t pocketsphinx_get_result(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request, mrcp_message_t *response)
{
if(pocketsphinx_result_build(recognizer,response) != TRUE) {
response->start_line.status_code = MRCP_STATUS_CODE_METHOD_FAILED;
}
/* send asynchronous response */
mrcp_engine_channel_message_send(recognizer->channel,response);
return TRUE;
}
/** Process START-INPUT-TIMERS request [RECOG] */
static apt_bool_t pocketsphinx_start_input_timers(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request, mrcp_message_t *response)
{
recognizer->is_input_timer_on = TRUE;
/* send asynchronous response */
mrcp_engine_channel_message_send(recognizer->channel,response);
return TRUE;
}
/** Process STOP request [RECOG] */
static apt_bool_t pocketsphinx_stop(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request, mrcp_message_t *response)
{
if(recognizer->inprogress_recog) {
/* store pending STOP response for further processing */
recognizer->stop_response = response;
return TRUE;
}
/* send asynchronous response */
mrcp_engine_channel_message_send(recognizer->channel,response);
return TRUE;
}
/** Process RECOGNITION-COMPLETE event [RECOG] */
static apt_bool_t pocketsphinx_recognition_complete(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *complete_event)
{
mrcp_recog_header_t *recog_header;
if(!recognizer->inprogress_recog) {
/* false event */
return FALSE;
}
recognizer->inprogress_recog = NULL;
ps_end_utt(recognizer->decoder);
if(recognizer->stop_response) {
/* recognition has been stopped, send STOP response instead */
mrcp_message_t *response = recognizer->stop_response;
recognizer->stop_response = NULL;
if(recognizer->close_requested == FALSE) {
mrcp_engine_channel_message_send(recognizer->channel,response);
}
return TRUE;
}
recog_header = mrcp_resource_header_get(complete_event);
if(recog_header->completion_cause == RECOGNIZER_COMPLETION_CAUSE_SUCCESS ||
recog_header->completion_cause == RECOGNIZER_COMPLETION_CAUSE_RECOGNITION_TIMEOUT) {
int32 score;
char const *hyp;
char const *uttid;
hyp = ps_get_hyp(recognizer->decoder, &score, &uttid);
if(hyp && strlen(hyp) > 0) {
int32 prob;
recognizer->last_result = apr_pstrdup(recognizer->channel->pool,hyp);
prob = ps_get_prob(recognizer->decoder, &uttid);
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Get Recognition Final Result [%s] Prob [%d] Score [%d] "APT_SIDRES_FMT,
hyp,prob,score,RECOGNIZER_SIDRES(recognizer));
if(pocketsphinx_result_build(recognizer,complete_event) != TRUE) {
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_ERROR;
}
}
else {
recog_header->completion_cause = RECOGNIZER_COMPLETION_CAUSE_NO_MATCH;
}
}
/* send asynchronous event */
mrcp_engine_channel_message_send(recognizer->channel,complete_event);
return TRUE;
}
/** Dispatch MRCP request [RECOG] */
static apt_bool_t pocketsphinx_request_dispatch(pocketsphinx_recognizer_t *recognizer, mrcp_message_t *request)
{
apt_bool_t processed = FALSE;
mrcp_message_t *response = mrcp_response_create(request,request->pool);
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Dispatch Request %s "APT_SIDRES_FMT,
request->start_line.method_name.buf,
RECOGNIZER_SIDRES(recognizer));
switch(request->start_line.method_id) {
case RECOGNIZER_SET_PARAMS:
break;
case RECOGNIZER_GET_PARAMS:
break;
case RECOGNIZER_DEFINE_GRAMMAR:
processed = pocketsphinx_define_grammar(recognizer,request,response);
break;
case RECOGNIZER_RECOGNIZE:
processed = pocketsphinx_recognize(recognizer,request,response);
break;
case RECOGNIZER_GET_RESULT:
processed = pocketsphinx_get_result(recognizer,request,response);
break;
case RECOGNIZER_START_INPUT_TIMERS:
processed = pocketsphinx_start_input_timers(recognizer,request,response);
break;
case RECOGNIZER_STOP:
processed = pocketsphinx_stop(recognizer,request,response);
@ -201,32 +679,200 @@ static apt_bool_t pocketsphinx_recognizer_request_process(mrcp_engine_channel_t
break;
}
if(processed == FALSE) {
/* send asynchronous response for not handled request */
mrcp_engine_channel_message_send(channel,response);
/* send asynchronous response for non handled request */
mrcp_engine_channel_message_send(recognizer->channel,response);
}
return TRUE;
}
static apt_bool_t pocketsphinx_stream_destroy(mpf_audio_stream_t *stream)
{
return TRUE;
}
static apt_bool_t pocketsphinx_stream_open(mpf_audio_stream_t *stream)
{
return TRUE;
}
static apt_bool_t pocketsphinx_stream_close(mpf_audio_stream_t *stream)
/** Recognition thread [RECOG] */
static void* APR_THREAD_FUNC pocketsphinx_recognizer_run(apr_thread_t *thread, void *data)
{
pocketsphinx_recognizer_t *recognizer = data;
apt_bool_t status;
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Run Recognition Thread "APT_SIDRES_FMT, RECOGNIZER_SIDRES(recognizer));
status = pocketsphinx_properties_load(recognizer);
/** Send response to channel_open request */
mrcp_engine_channel_open_respond(recognizer->channel,status);
do {
/** Wait for MRCP requests */
apt_log(APT_LOG_MARK,APT_PRIO_DEBUG,"Wait for incoming messages "APT_SIDRES_FMT, RECOGNIZER_SIDRES(recognizer));
apr_thread_mutex_lock(recognizer->mutex);
apr_thread_cond_wait(recognizer->wait_object,recognizer->mutex);
apr_thread_mutex_unlock(recognizer->mutex);
if(recognizer->request) {
/* store request message and further dispatch it */
mrcp_message_t *request = recognizer->request;
recognizer->request = NULL;
pocketsphinx_request_dispatch(recognizer,request);
}
if(recognizer->complete_event) {
/* end of input detected, get recognition result and raise recognition complete event */
pocketsphinx_recognition_complete(recognizer,recognizer->complete_event);
}
}
while(recognizer->close_requested == FALSE);
/* check if recognition is still active */
if(recognizer->inprogress_recog) {
apr_thread_mutex_lock(recognizer->mutex);
recognizer->stop_response = recognizer->inprogress_recog;
apr_thread_cond_wait(recognizer->wait_object,recognizer->mutex);
apr_thread_mutex_unlock(recognizer->mutex);
if(recognizer->complete_event) {
pocketsphinx_recognition_complete(recognizer,recognizer->complete_event);
}
}
/** Clear all the defined grammars */
pocketsphinx_grammars_clear(recognizer);
if(recognizer->decoder) {
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Free Decoder "APT_SIDRES_FMT, RECOGNIZER_SIDRES(recognizer));
/** Free pocketsphinx decoder */
ps_free(recognizer->decoder);
recognizer->decoder = NULL;
}
/** Exit thread */
apr_thread_exit(thread,APR_SUCCESS);
return NULL;
}
/* Start of input (utterance) [MPF] */
static apt_bool_t pocketsphinx_start_of_input(pocketsphinx_recognizer_t *recognizer)
{
/* create START-OF-INPUT event */
mrcp_message_t *message = mrcp_event_create(
recognizer->inprogress_recog,
RECOGNIZER_START_OF_INPUT,
recognizer->inprogress_recog->pool);
if(!message) {
return FALSE;
}
/* set request state */
message->start_line.request_state = MRCP_REQUEST_STATE_INPROGRESS;
/* send asynch event */
return mrcp_engine_channel_message_send(recognizer->channel,message);
}
/* End of input (utterance) [MPF] */
static apt_bool_t pocketsphinx_end_of_input(pocketsphinx_recognizer_t *recognizer, mrcp_recog_completion_cause_e cause)
{
mrcp_recog_header_t *recog_header;
/* create RECOGNITION-COMPLETE event */
mrcp_message_t *message = mrcp_event_create(
recognizer->inprogress_recog,
RECOGNIZER_RECOGNITION_COMPLETE,
recognizer->inprogress_recog->pool);
if(!message) {
return FALSE;
}
/* get/allocate recognizer header */
recog_header = mrcp_resource_header_prepare(message);
if(recog_header) {
/* set completion cause */
recog_header->completion_cause = cause;
mrcp_resource_header_property_add(message,RECOGNIZER_HEADER_COMPLETION_CAUSE);
}
/* set request state */
message->start_line.request_state = MRCP_REQUEST_STATE_COMPLETE;
/* signal recognition thread first */
apr_thread_mutex_lock(recognizer->mutex);
recognizer->complete_event = message;
apr_thread_cond_signal(recognizer->wait_object);
apr_thread_mutex_unlock(recognizer->mutex);
return TRUE;
}
/* Process audio frame [MPF] */
static apt_bool_t pocketsphinx_stream_write(mpf_audio_stream_t *stream, const mpf_frame_t *frame)
{
pocketsphinx_recognizer_t *recognizer = stream->obj;
/* check whether recognition has been started and not completed yet */
if(recognizer->inprogress_recog && !recognizer->complete_event) {
mpf_detector_event_e det_event;
/* first check if STOP has been requested */
if(recognizer->stop_response) {
/* recognition has been stopped -> acknowledge with complete-event */
pocketsphinx_end_of_input(recognizer,RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
return TRUE;
}
if(ps_process_raw(
recognizer->decoder,
(const int16 *)frame->codec_frame.buffer,
frame->codec_frame.size / sizeof(int16),
FALSE,
FALSE) < 0) {
apt_log(APT_LOG_MARK,APT_PRIO_WARNING,"Failed to Process Raw Data "APT_SIDRES_FMT,
RECOGNIZER_SIDRES(recognizer));
}
recognizer->partial_result_timeout += CODEC_FRAME_TIME_BASE;
if(recognizer->partial_result_timeout == recognizer->properties.partial_result_timeout) {
int32 score;
char const *hyp;
char const *uttid;
recognizer->partial_result_timeout = 0;
hyp = ps_get_hyp(recognizer->decoder, &score, &uttid);
if(hyp && strlen(hyp) > 0) {
if(recognizer->last_result == NULL || 0 != strcmp(recognizer->last_result, hyp)) {
recognizer->last_result = apr_pstrdup(recognizer->channel->pool,hyp);
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Get Recognition Partial Result [%s] Score [%d] "APT_SIDRES_FMT,
hyp,score,RECOGNIZER_SIDRES(recognizer));
}
}
}
if(recognizer->is_input_timer_on) {
recognizer->noinput_timeout += CODEC_FRAME_TIME_BASE;
if(recognizer->noinput_timeout == recognizer->properties.noinput_timeout) {
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Noinput Timeout Elapsed "APT_SIDRES_FMT,
RECOGNIZER_SIDRES(recognizer));
pocketsphinx_end_of_input(recognizer,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);
return TRUE;
}
}
recognizer->recognition_timeout += CODEC_FRAME_TIME_BASE;
if(recognizer->recognition_timeout == recognizer->properties.recognition_timeout) {
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Recognition Timeout Elapsed "APT_SIDRES_FMT,
RECOGNIZER_SIDRES(recognizer));
pocketsphinx_end_of_input(recognizer,RECOGNIZER_COMPLETION_CAUSE_RECOGNITION_TIMEOUT);
return TRUE;
}
det_event = mpf_activity_detector_process(recognizer->detector,frame);
switch(det_event) {
case MPF_DETECTOR_EVENT_ACTIVITY:
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Detected Voice Activity "APT_SIDRES_FMT,
RECOGNIZER_SIDRES(recognizer));
pocketsphinx_start_of_input(recognizer);
break;
case MPF_DETECTOR_EVENT_INACTIVITY:
apt_log(APT_LOG_MARK,APT_PRIO_INFO,"Detected Voice Inactivity "APT_SIDRES_FMT,
RECOGNIZER_SIDRES(recognizer));
pocketsphinx_end_of_input(recognizer,RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
break;
default:
break;
}
}
return TRUE;
}

@ -27,27 +27,27 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcp", "libs\mrcp\mrcp.vcpr
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpsignaling", "libs\mrcp-signaling\mrcpsignaling.vcproj", "{12A49562-BAB9-43A3-A21D-15B60BBB4C31}"
ProjectSection(ProjectDependencies) = postProject
{B5A00BFA-6083-4FAE-A097-71642D6473B5} = {B5A00BFA-6083-4FAE-A097-71642D6473B5}
{1C320193-46A6-4B34-9C56-8AB584FC1B56} = {1C320193-46A6-4B34-9C56-8AB584FC1B56}
{B5A00BFA-6083-4FAE-A097-71642D6473B5} = {B5A00BFA-6083-4FAE-A097-71642D6473B5}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpserver", "libs\mrcp-server\mrcpserver.vcproj", "{18B1F35A-10F8-4287-9B37-2D10501B0B38}"
ProjectSection(ProjectDependencies) = postProject
{12A49562-BAB9-43A3-A21D-15B60BBB4C31} = {12A49562-BAB9-43A3-A21D-15B60BBB4C31}
{A9EDAC04-6A5F-4BA7-BC0D-CCE7B255B6EA} = {A9EDAC04-6A5F-4BA7-BC0D-CCE7B255B6EA}
{12A49562-BAB9-43A3-A21D-15B60BBB4C31} = {12A49562-BAB9-43A3-A21D-15B60BBB4C31}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libunimrcpserver", "platforms\libunimrcp-server\libunimrcpserver.vcproj", "{C98AF157-352E-4737-BD30-A24E2647F5AE}"
ProjectSection(ProjectDependencies) = postProject
{DEB01ACB-D65F-4A62-AED9-58C1054499E9} = {DEB01ACB-D65F-4A62-AED9-58C1054499E9}
{18B1F35A-10F8-4287-9B37-2D10501B0B38} = {18B1F35A-10F8-4287-9B37-2D10501B0B38}
{746F3632-5BB2-4570-9453-31D6D58A7D8E} = {746F3632-5BB2-4570-9453-31D6D58A7D8E}
{18B1F35A-10F8-4287-9B37-2D10501B0B38} = {18B1F35A-10F8-4287-9B37-2D10501B0B38}
{DEB01ACB-D65F-4A62-AED9-58C1054499E9} = {DEB01ACB-D65F-4A62-AED9-58C1054499E9}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpsofiasip", "modules\mrcp-sofiasip\mrcpsofiasip.vcproj", "{746F3632-5BB2-4570-9453-31D6D58A7D8E}"
ProjectSection(ProjectDependencies) = postProject
{12A49562-BAB9-43A3-A21D-15B60BBB4C31} = {12A49562-BAB9-43A3-A21D-15B60BBB4C31}
{A9EDAC04-6A5F-4BA7-BC0D-CCE7B255B6EA} = {A9EDAC04-6A5F-4BA7-BC0D-CCE7B255B6EA}
{12A49562-BAB9-43A3-A21D-15B60BBB4C31} = {12A49562-BAB9-43A3-A21D-15B60BBB4C31}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpclient", "libs\mrcp-client\mrcpclient.vcproj", "{72782932-37CC-46AE-8C7F-9A7B1A6EE108}"
@ -57,9 +57,9 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpclient", "libs\mrcp-cli
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libunimrcpclient", "platforms\libunimrcp-client\libunimrcpclient.vcproj", "{EE157390-1E85-416C-946E-620E32C9AD33}"
ProjectSection(ProjectDependencies) = postProject
{DEB01ACB-D65F-4A62-AED9-58C1054499E9} = {DEB01ACB-D65F-4A62-AED9-58C1054499E9}
{746F3632-5BB2-4570-9453-31D6D58A7D8E} = {746F3632-5BB2-4570-9453-31D6D58A7D8E}
{72782932-37CC-46AE-8C7F-9A7B1A6EE108} = {72782932-37CC-46AE-8C7F-9A7B1A6EE108}
{746F3632-5BB2-4570-9453-31D6D58A7D8E} = {746F3632-5BB2-4570-9453-31D6D58A7D8E}
{DEB01ACB-D65F-4A62-AED9-58C1054499E9} = {DEB01ACB-D65F-4A62-AED9-58C1054499E9}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "unimrcpclient", "platforms\unimrcp-client\unimrcpclient.vcproj", "{57FAF32E-49FD-491F-895D-132D0D5EFE0A}"
@ -74,8 +74,8 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpv2transport", "libs\mrc
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpengine", "libs\mrcp-engine\mrcpengine.vcproj", "{843425BE-9A9A-44F4-A4E3-4B57D6ABD53C}"
ProjectSection(ProjectDependencies) = postProject
{B5A00BFA-6083-4FAE-A097-71642D6473B5} = {B5A00BFA-6083-4FAE-A097-71642D6473B5}
{1C320193-46A6-4B34-9C56-8AB584FC1B56} = {1C320193-46A6-4B34-9C56-8AB584FC1B56}
{B5A00BFA-6083-4FAE-A097-71642D6473B5} = {B5A00BFA-6083-4FAE-A097-71642D6473B5}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "demosynth", "plugins\demo-synth\demosynth.vcproj", "{92BFA534-C419-4EB2-AAA3-510653F38F08}"
@ -127,8 +127,8 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "rtsptest", "tests\rtsptest\
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpunirtsp", "modules\mrcp-unirtsp\mrcpunirtsp.vcproj", "{DEB01ACB-D65F-4A62-AED9-58C1054499E9}"
ProjectSection(ProjectDependencies) = postProject
{12A49562-BAB9-43A3-A21D-15B60BBB4C31} = {12A49562-BAB9-43A3-A21D-15B60BBB4C31}
{504B3154-7A4F-459D-9877-B951021C3F1F} = {504B3154-7A4F-459D-9877-B951021C3F1F}
{12A49562-BAB9-43A3-A21D-15B60BBB4C31} = {12A49562-BAB9-43A3-A21D-15B60BBB4C31}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "prepare", "build\tools\prepare.vcproj", "{01D63BF5-7798-4746-852A-4B45229BB735}"
@ -138,6 +138,16 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "unimrcpservice", "build\too
{13DEECA0-BDD4-4744-A1A2-8EB0A44DF3D2} = {13DEECA0-BDD4-4744-A1A2-8EB0A44DF3D2}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcppocketsphinx", "plugins\mrcp-pocketsphinx\mrcppocketsphinx.vcproj", "{3C614AE8-B611-4D43-A9AF-1CAA440A9F69}"
ProjectSection(ProjectDependencies) = postProject
{843425BE-9A9A-44F4-A4E3-4B57D6ABD53C} = {843425BE-9A9A-44F4-A4E3-4B57D6ABD53C}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mrcpflite", "plugins\mrcp-flite\mrcpflite.vcproj", "{56F6FB96-2BC7-4CAE-A8BF-6A0FAEC90556}"
ProjectSection(ProjectDependencies) = postProject
{843425BE-9A9A-44F4-A4E3-4B57D6ABD53C} = {843425BE-9A9A-44F4-A4E3-4B57D6ABD53C}
EndProjectSection
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
@ -238,6 +248,10 @@ Global
{01D63BF5-7798-4746-852A-4B45229BB735}.Release|Win32.ActiveCfg = Release|Win32
{4714EF49-BFD5-4B22-95F7-95A07F1EAC25}.Debug|Win32.ActiveCfg = Debug|Win32
{4714EF49-BFD5-4B22-95F7-95A07F1EAC25}.Release|Win32.ActiveCfg = Release|Win32
{3C614AE8-B611-4D43-A9AF-1CAA440A9F69}.Debug|Win32.ActiveCfg = Debug|Win32
{3C614AE8-B611-4D43-A9AF-1CAA440A9F69}.Release|Win32.ActiveCfg = Release|Win32
{56F6FB96-2BC7-4CAE-A8BF-6A0FAEC90556}.Debug|Win32.ActiveCfg = Debug|Win32
{56F6FB96-2BC7-4CAE-A8BF-6A0FAEC90556}.Release|Win32.ActiveCfg = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
@ -261,6 +275,8 @@ Global
{92BFA534-C419-4EB2-AAA3-510653F38F08} = {09BABD45-8F30-4F99-B8B8-8DD78F6804DB}
{B495B6D9-AF84-479D-B30A-313C16EF8BFD} = {09BABD45-8F30-4F99-B8B8-8DD78F6804DB}
{729EF28E-38C9-40DE-A138-87785F021411} = {09BABD45-8F30-4F99-B8B8-8DD78F6804DB}
{3C614AE8-B611-4D43-A9AF-1CAA440A9F69} = {09BABD45-8F30-4F99-B8B8-8DD78F6804DB}
{56F6FB96-2BC7-4CAE-A8BF-6A0FAEC90556} = {09BABD45-8F30-4F99-B8B8-8DD78F6804DB}
{79EF9F1D-E211-4ED1-91D2-FC935AB3A872} = {AC4356E8-48A1-4D2D-AFB1-11CF30B974CD}
{429C907B-97D1-4B2D-9B0E-A14A5BFDAD15} = {AC4356E8-48A1-4D2D-AFB1-11CF30B974CD}
{DCF01B1C-5268-44F3-9130-D647FABFB663} = {AC4356E8-48A1-4D2D-AFB1-11CF30B974CD}