AN02031: Live Streaming Sound Card Example£££doc/rst/an02031.html#an02031-live-streaming-sound-card-example

AN02031: Live Streaming Sound Card Example$$$Introduction£££doc/rst/an02031.html#introduction

This application note demonstrates how to use lib_audio_dsp to create a live streaming sound card application. The device captures sound from both a USB source and a microphone, outputting through USB and analog outputs. The sound is processed through a series of Digital Signal Processing (DSP) stages, which can be tuned on the host machine. The application includes a Graphical User Interface (GUI) that allow these adjustments.

Note

Some software components in this tool flow are prototypes and will be updated in Version 2 of the library. The underlying Digital Signal Processing (DSP) blocks are however fully functional. Future updates will enhance the features and flexibility of the design tool.

The sound card features the following DSP stages:

It could be used for applications such as:

The application has the following inputs and outputs:

The DSP pipeline is shown below:

../../_images/pipeline.png

DSP Pipeline

The audio performance of the demo is:

AN02031: Live Streaming Sound Card Example$$$Setup£££doc/rst/an02031.html#setup

This section guides the user through installing all the necessary components for setting up the application, from both a software and hardware point of view.

AN02031: Live Streaming Sound Card Example$$$Setup$$$Prerequisites£££doc/rst/an02031.html#prerequisites

The following are required before setting up the application:

AN02031: Live Streaming Sound Card Example$$$Setup$$$Hardware Setup£££doc/rst/an02031.html#hardware-setup

Connect the XMOS MCAB to the host computer using the USB DEVICE port. For flashing the board, also connect the DEBUG port. The DEBUG port can be disconnected once the device has been flashed.

Connect a microphone to the IN 1/2 jack, and headphones to the OUT 1/2 jack.

Note

The microphone input is a stereo TRS jack. This means a balanced mic should not be used, as the stereo inputs are summed, and for a balanced signal this results in total signal cancellation.

Note

A low-impedance microphone or one with a powered preamplifier is recommended to ensure a strong signal for processing. Otherwise, the signal may be too weak for proper handling. Alternatively, users can manually add a fixed gain stage near the beginning of the pipeline.

See the Fig. 2 for an example hardware setup.

../../_images/mcab_setup_photo.jpg

USB Multichannel Audio Evaluation Kit Setup

AN02031: Live Streaming Sound Card Example$$$Setup$$$Software Setup£££doc/rst/an02031.html#software-setup

A compiled binary is provided, and the device can be flashed from a XMOS XTC command prompt using:

xflash app_an02031\bin\app_an02031.xe

To build from source, refer to the section Saving, Loading and Code Generation.

AN02031: Live Streaming Sound Card Example$$$Setup$$$Installing the libusb Driver on Windows£££doc/rst/an02031.html#installing-the-libusb-driver-on-windows

The first time the device is used on Windows, the libusb driver must be installed. This is done using the third-party tool Zadig.

These steps are only required once and they must be executed while the firmware is running on the device.

  1. Open Zadig and select XMOS Control (Interface 3) from the list of devices. If the device is not present, ensure Options -> List All Devices is checked.

  2. Select libusb-win32 from the list of drivers in the right hand spin box as shown in Fig. 3.

  3. Click the Install Driver button and wait for the installation to complete.

../../_images/zadig_install_control.png

Selecting the libusb-win32 driver in Zadig for the Control Interface

AN02031: Live Streaming Sound Card Example$$$Running the GUI£££doc/rst/an02031.html#running-the-gui

AN02031: Live Streaming Sound Card Example$$$Running the GUI$$$Initial Use£££doc/rst/an02031.html#initial-use

In Windows Explorer, double click on tuning.cmd. This will initialise the Python virtual environment and install requirements. Once this is done it will open the GUI.

Note

Customers in China may wish to use tuning_cn.cmd instead. This will use https://pypi.tuna.tsinghua.edu.cn/simple for retrieving Python packages instead of https://pypi.python.org/simple, which may give increased download speeds.

The GUI has 4 tabs described below.

AN02031: Live Streaming Sound Card Example$$$Running the GUI$$$End Customer Interface£££doc/rst/an02031.html#end-customer-interface

These are the controls exposed to the final customer. In this implementation, USB control commands are used over EP0. The same control logic could alternatively be implemented via GPIO.

../../_images/gui_tab_1.png

GUI Control Tab

The controls perform the following functions:

  • Volume sliders: control the volume of the input/output with optional mute.

  • Reverb slider: control wet/dry mix of the reverb.

  • Reverb button: enable/disable the reverb.

  • Denoise button: enable noise suppression on the microphone.

  • Ducking button: reduce the volume of the music when there is signal on the microphone.

  • Monitor button: when on, send the same output to the USB and headphone outputs. When off, do not send the microphone output to the headphones.

  • Loopback button: when on, send the USB input back to the USB output. Otherwise just send the microphone signal over USB.

AN02031: Live Streaming Sound Card Example$$$Running the GUI$$$Algorithm Tuning Parameters£££doc/rst/an02031.html#algorithm-tuning-parameters

This tab exposes the lower level tuning parameters to a DSP product engineer for tuning before final production.

../../_images/gui_tab_2.png

GUI Configuration Tab

AN02031: Live Streaming Sound Card Example$$$Running the GUI$$$Parametric EQ£££doc/rst/an02031.html#parametric-eq

This tab shows the EQ being applied to the microphone signal. The type and parameters of each biquad can be set, and the frequency response of the cascade is calculated from it.

../../_images/gui_tab_3.png

GUI Equaliser Tab

AN02031: Live Streaming Sound Card Example$$$Running the GUI$$$Pipeline Graphic£££doc/rst/an02031.html#pipeline-graphic

This tab contains an image of the DSP pipeline.

../../_images/gui_tab_4.png

GUI Pipeline Tab

AN02031: Live Streaming Sound Card Example$$$Saving, Loading and Code Generation£££doc/rst/an02031.html#saving-loading-and-code-generation

Once the parameters have been tuned to the desired values, they can be saved as a JSON file by clicking the “Save” button. Other sets of parameters can be loaded from the JSON file by using the “Load” button. During loading, the saved checksum in the JSON file is compared against the checksum generated from the pipeline defined in pipeline.py. If they do not match, a warning is given to indicate that the pipelines differ. The loaded JSON file may still be compatible with the GUI if the exposed tuning parameters defined in params.py have remained the same, and the DSP stages they interact with still exist. If the loaded JSON file is not compatible with the current version of the GUI an error is given. This usually means that pipeline.py and params.py have been changed, and a different version of the GUI needs to be used.

Once the parameters are fully configured, the autogenerated C should be updated by clicking the “Generate Code” button in the GUI. By default this selects the folder app_an02031\src\generated_dsp. The application can then be rebuild by following the steps in Building the App from Source.

Alternatively the code can be generated from the command line, see the steps in Updating the DSP Pipeline. This may be useful for regenerating the code as part of a CMake step.

AN02031: Live Streaming Sound Card Example$$$Building the App from Source£££doc/rst/an02031.html#building-the-app-from-source

The app is supplied with a full sandbox and compiled app. Modifications may require rebuilding, which can be done as described below.

In an XTC Tools 15.3 prompt, navigate to the app note directory, then create and activate a Python virtual environment, and install lib_audio_dsp:

python -m venv .venv
.venv\Scripts\activate
pip install -e lib_audio_dsp\python

Then run Xcommon Cmake to initialise the sandbox:

cd an02031
cmake -G "Unix Makefiles" -B build

To build the app, run:

cmake --build build

Finally, to run it:

xrun app_an02031\bin\app_an02031.xe

# Or, to permanently flash it
xflash app_an02031\bin\app_an02031.xe

AN02031: Live Streaming Sound Card Example$$$Updating the GUI or Modifying the DSP£££doc/rst/an02031.html#updating-the-gui-or-modifying-the-dsp

This application note provides a fixed DSP pipeline and a GUI designed to interact with it. Users may wish to modify or extend the provided DSP pipeline and correspondingly update the GUI. The following sections detail the procedures for accomplishing these tasks.

AN02031: Live Streaming Sound Card Example$$$Updating the GUI or Modifying the DSP$$$Updating the DSP Pipeline£££doc/rst/an02031.html#updating-the-dsp-pipeline

Should it be necessary, the first step involves updating the DSP pipeline. The pipeline is defined in the file tuning_utility\pipeline.py and utilizes the lib_audio_dsp Python library to generate a pipeline. The library includes its own documentation, which can be consulted to implement any required modifications.

Prior to generating sources, it may be necessary to update the parameter model, which is defined in tuning_utility\params.py. This file defines a class named Params that contains the tuning data for the named stages in the pipeline. Initially, “tuning_gui.py” will invoke pipeline() to create an instance of Pipeline. Subsequently, it will instantiate Params and invoke the function update_from_tuning_params from “pipeline.py” to adjust the pipeline with the tuning parameters.

When updating the pipeline, it is likely the checksum will change. This may give errors when trying to reuse a tuning.json file. Removing or renaming the default tuning.json should resolve most issues. Note that if the pipeline is different, but the params.py is the same, the new pipeline may be compatible with the GUI despite the pipeline difference.

To regenerate the sources for the updated pipeline, create a Python virtual environment and install the required dependencies specified in “requirements.txt” using pip. Then, execute the following command to generate the code for the default JSON file:

python tuning_gui.py --code-gen

To use a different JSON file, the path to the file can be passed in:

python tuning_gui.py --code-gen path/to/tuning.json

This command will update the generated DSP sources. Subsequently, use CMake to compile the new application in the usual manner.

AN02031: Live Streaming Sound Card Example$$$Updating the GUI or Modifying the DSP$$$Updating the GUI£££doc/rst/an02031.html#updating-the-gui

This section addresses the addition or removal of widgets in the GUI. The GUI is a straightforward Python application utilizing PySide, a Python binding for the Qt framework. The GUI is defined in “tuning_gui.py”.

Within “tuning_gui.py”, the DSPWindow class is defined. This class is a QMainWindow to which all widgets are added. The DSPWindow class contains a state attribute that tracks the current parameter values. All widgets must update this state through one of its update methods. To ensure the widget remains current when the value it controls may be updated elsewhere, it should connect to the STATE_CHANGED signal. This signal is emitted whenever any parameters change.