Tags: Accessibility and Inclusivity, Artificial Intelligence, Audio Intelligence, Collaboration Solutions, Collaboration Tools, Gesture Detection, Optimize for my Voice, Real-Time Closed Captioning, Video Intelligence, Webex App Features, Webex Assistant, Webex Hybrid Security, Workplace Transformation
Webex video conferencing devices and the Webex App provide far greater functionality than solely voice and video streaming. Webex devices and the Webex App employ collaboration AI software that make them easier to use and provide value added functions by using the device’s microphone, speaker and camera. This blog provides details on how these advanced collaboration features work and explains how these features were developed with your privacy in mind.
Webex devices and the Webex App deliver far more than great audio and video for calls and meetings; we have developed a range of amazing AI features that can enhance and improve your experience when in a Webex call or Webex meeting:
At first glance, one might assume that some of these advanced collaboration features leverage collaboration AI by silently streaming audio and video to the Webex cloud for processing. In fact, most of these advanced Webex features process audio and video locally. Webex only streams media to the cloud when you are in a call or meeting; we do not listen to users when they are not participating in one of these.
The section below describes how advanced Webex collaboration features work. The table below describes whether the feature is supported by Webex devices and/or the Webex App, if media is streamed to the Webex cloud to support this feature and how the feature can be enabled/ disabled.
Audio intelligence is only used during a call or meeting and audio is processed locally on the device. Background noise reduction removes unwanted noise such as keyboard chatter, a dog barking, etc. Audio optimization features are used to suppress the voices of those speaking in the background, or to optimize audio for music. If you have muted your microphone during a call, we do maintain access to the microphone and sample the audio to determine if you are speaking, so we can prompt you to let you know that your microphone is muted. Audio samples are never sent to the Webex cloud when you are muted.
Only used during Webex Meetings and Webinars, gesture recognition uses the camera and onboard software to detect when meeting participants raise their hands, give a thumbs up or thumbs down, or clap. These gestures are momentarily presented as on-screen icons to participants in the meeting. For more information on using gesture recognition with a Webex Desk Series, click here.
Facial recognition is used to create and display name labels of meeting participants and is particularly useful when multiple participants are in the same room. When facial recognition is enabled for your organization, your Webex device uses its camera and software to create a vector data set that represents your face. This data set is sent to the face recognition service in the Webex cloud. We only detect facial images when you are an active participant in a meeting, and we will only detect your face and provide a name label if you have opted-into the feature. Our facial recognition technology never streams your actual image to the face recognition system. Getting started with Facial Recognition is simple, just sign up and enable the feature in Control Hub!
Language intelligence is used by Webex Assistant and in Webex Meetings for closed captioning, realtime translation and meeting transcription.
Webex Assistant for Devices is activated by the wake word “OK Webex” or “Hey Webex.” Once activated, you can give Webex Assistant verbal commands such as “start meeting,” “increase volume,” or “call a number.” When using Webex Assistant, the Webex device is listening only for the wake word, which is processed locally. We do not listen by continuously streaming media to the Webex cloud. Hosts can also activate Webex Assistant in a meeting. Once you have activated Webex Assistant, we will send your verbal instructions in an audio stream to the Webex cloud for interpretation and execution.
Additionally, to make your meetings and webinars more accessible, Webex provides live automated closed captions which you can turn on without needing to turn on Webex Assistant. As people speak, the captions will appear above the meeting or webinar controls. If Webex Assistant is also on, then users can also make highlights or use voice commands, as described above.
For a comparison of Webex Assistant and closed captioning features, click here.
Webex Meetings also uses Language intelligence for:
Typically used during a call or meeting, people count uses the Webex device camera and software for head detection and sends the results to the Webex cloud. We do not keep a record of who was in the room, only the average number of people detected. If needed, you can choose to enable the count people feature outside of a call or meeting.
Your Webex device uses its speaker to emit an ultrasound audio signature and uses its own microphone to monitor the returned audio for changes. People presence doesn’t count people, but in theory if someone enters a room, even off camera, this can detect they are there.
Ultrasound pairing allows your Webex App to discover, pair with, and control nearby Webex devices (Board & Room Series), so that you can make calls and share your screen on the device. Webex devices use ultrasound to transmit beacons to nearby Webex Apps. The Webex App on your personal computer, or mobile device, uses its microphone to detect these ultrasound beacons. With ultrasound pairing enabled, the microphone used by the Webex App will be enabled to listen for beacons, but no audio is sent to the Webex cloud. The Webex Rooms Security whitepaper provides a description of how ultrasound pairing works, along with other Webex Room security features.
Wi-fi based device discovery and pairing allows your Webex App to discover, pair with, and control nearby Webex devices (Board & Room series), so that you can join meetings, make calls and share your screen on the device.
The table below describes each of the features listed above, their basic operation
|Webex Advanced Collaboration Feature||Supported by||Audio or Video sent to Webex cloud for this feature?||Where this feature can be configured||Default Setting|
|Background noise removal||Webex devices|
|No||End User settings||On|
|Optimize for my voice/all voices||Webex App||No||End User settings||Off|
|Music Mode||Webex Devices|
|No||End User settings||Off|
|Gesture Recognition||Webex Devices|
|No||End User In-meeting controls||On|
|Face Recognition||Webex Devices||No||Webex administrator (global setting for organization) and End User settings||Org: Off|
Webex Assistant (WXA)
|Yes – But only when activated using the wake word, or when activated in a meeting.||Webex administrator (Organization and Site settings) and End User In-meeting controls||Site: Off|
Real Time Translation
|Webex devices Webex App||Yes – Used during a Webex Meeting||Webex administrator (Site setting) and End user In-meeting controls||Org: Off|
|Webex devices Webex App||Uses AI to convert meeting recordings into transcripts||Webex administrator (global setting for organization) and End User settings||Org: On|
|Webex devices |
|Yes – Used during a Webex Meeting||Webex administrator (global setting for organization) and End User In-meeting controls||Org: On|
|Room interpretation – People count||Webex devices||No||Webex administrator (global setting for organization)||Org: Off|
In call: Off
Out of call: Off
|People presence detection||Webex devices||No||Webex administrator (global setting for organization)||Org: Off|
|Ultrasound based pairing||Webex devices|
|No||Webex administrator (global setting for organization) Webex device settings Webex App: End User settings||Org: On|
Desktop app: On
Mobile app: Off
|WiFi based pairing||Webex devices Webex App (Desktop only)||No||Webex device settings Webex App (Desktop): End User settings||Org : On Device: On|
Cisco considers privacy to be a fundamental human right, and we employ a responsible approach to developing AI. We are open and transparent about the personal data, including within user-generated content and metadata, that we collect and store in the Webex cloud when you use our services. Personal data protection and privacy considerations forms part of the design and development process for Cisco products and services. As such, all Cisco products and services undergo rigorous Privacy Impact Assessments prior to launch. For additional information on our personal information practices with respect to Cisco offers see the Webex App and Webex Meetings Privacy datasheets on Cisco’s Trust Portal.
More about Webex devices: https://hardware.webex.com/products
More about the Webex App: https://www.webex.com/all-new-webex.html