Webex Advanced Collaboration features Providing Amazing User Experiences while Protecting Privacy

Webex video conferencing devices and the Webex App provide far greater functionality than solely voice and video streaming. Webex devices and the Webex App employ collaboration AI software that make them easier to use and provide value added functions by using the device’s microphone, speaker and camera. This blog provides details on how these advanced collaboration features work and explains how these features were developed with your privacy in mind.

How Webex is different

Webex devices and the Webex App deliver far more than great audio and video for calls and meetings; we have developed a range of amazing AI features that can enhance and improve your experience when in a Webex call or Webex meeting:

• Background noise removal
• Optimize for my voice/all voices
• Music mode
• Gesture recognition
• Face recognition
• Language intelligence:
  • Webex Assistant
  • Closed captioning
  • Real time translation
  • Meeting transcription
• Room interpretation
• People presence detection
• Proximity Pairing

At first glance, one might assume that some of these advanced collaboration features leverage collaboration AI by silently streaming audio and video to the Webex cloud for processing. In fact, most of these advanced Webex features process audio and video locally. Webex only streams media to the cloud when you are in a call or meeting; we do not listen to users when they are not participating in one of these.

Webex Advanced collaboration features and your privacy

The section below describes how advanced Webex collaboration features work. The table below describes whether the feature is supported by Webex devices and/or the Webex App, if media is streamed to the Webex cloud to support this feature and how the feature can be enabled/ disabled.

Webex Audio Intelligence: Background Noise Removal & Audio Optimization

Audio intelligence is only used during a call or meeting and audio is processed locally on the device. Background noise reduction removes unwanted noise such as keyboard chatter, a dog barking, etc. Audio optimization features are used to suppress the voices of those speaking in the background, or to optimize audio for music. If you have muted your microphone during a call, we do maintain access to the microphone and sample the audio to determine if you are speaking, so we can prompt you to let you know that your microphone is muted. Audio samples are never sent to the Webex cloud when you are muted.

Gesture Recognition

Only used during Webex Meetings and Webinars, gesture recognition uses the camera and onboard software to detect when meeting participants raise their hands, give a thumbs up or thumbs down, or clap. These gestures are momentarily presented as on-screen icons to participants in the meeting. For more information on using gesture recognition with a Webex Desk Series, click here.

Facial Recognition

Facial recognition is used to create and display name labels of meeting participants and is particularly useful when multiple participants are in the same room. When facial recognition is enabled for your organization, your Webex device uses its camera and software to create a vector data set that represents your face. This data set is sent to the face recognition service in the Webex cloud. We only detect facial images when you are an active participant in a meeting, and we will only detect your face and provide a name label if you have opted-into the feature. Our facial recognition technology never streams your actual image to the face recognition system. Getting started with Facial Recognition is simple, just sign up and enable the feature in Control Hub!

Language Intelligence

Language intelligence is used by Webex Assistant and in Webex Meetings for closed captioning, realtime translation and meeting transcription.

Webex Assistant for Devices is activated by the wake word “OK Webex” or “Hey Webex.” Once activated, you can give Webex Assistant verbal commands such as “start meeting,” “increase volume,” or “call a number.” When using Webex Assistant, the Webex device is listening only for the wake word, which is processed locally. We do not listen by continuously streaming media to the Webex cloud. Hosts can also activate Webex Assistant in a meeting. Once you have activated Webex Assistant, we will send your verbal instructions in an audio stream to the Webex cloud for interpretation and execution.

Additionally, to make your meetings and webinars more accessible, Webex provides live automated closed captions which you can turn on without needing to turn on Webex Assistant. As people speak, the captions will appear above the meeting or webinar controls. If Webex Assistant is also on, then users can also make highlights or use voice commands, as described above.

For a comparison of Webex Assistant and closed captioning features, click here.

Webex Meetings also uses Language intelligence for:

• Real-time translation for multiple languages
• Meeting transcription

Room Interpretation – People Count

Typically used during a call or meeting, people count uses the Webex device camera and software for head detection and sends the results to the Webex cloud. We do not keep a record of who was in the room, only the average number of people detected. If needed, you can choose to enable the count people feature outside of a call or meeting.

People Presence Detection.

Your Webex device uses its speaker to emit an ultrasound audio signature and uses its own microphone to monitor the returned audio for changes. People presence doesn’t count people, but in theory if someone enters a room, even off camera, this can detect they are there.

Ultrasound Pairing

Ultrasound pairing allows your Webex App to discover, pair with, and control nearby Webex devices (Board & Room Series), so that you can make calls and share your screen on the device. Webex devices use ultrasound to transmit beacons to nearby Webex Apps. The Webex App on your personal computer, or mobile device, uses its microphone to detect these ultrasound beacons. With ultrasound pairing enabled, the microphone used by the Webex App will be enabled to listen for beacons, but no audio is sent to the Webex cloud. The Webex Rooms Security whitepaper provides a description of how ultrasound pairing works, along with other Webex Room security features.

Wi-fi based device discovery and pairing

Wi-fi based device discovery and pairing allows your Webex App to discover, pair with, and control nearby Webex devices (Board & Room series), so that you can join meetings, make calls and share your screen on the device.

The table below describes each of the features listed above, their basic operation

Webex Advanced Collaboration Feature	Supported by	Audio or Video sent to Webex cloud for this feature?	Where this feature can be configured	Default Setting
Background noise removal	Webex devices Webex App	No	End User settings	On
Optimize for my voice/all voices	Webex App	No	End User settings	Off
Music Mode	Webex Devices Webex App	No	End User settings	Off
Gesture Recognition	Webex Devices Webex App	No	End User In-meeting controls	On
Face Recognition	Webex Devices	No	Webex administrator (global setting for organization) and End User settings	Org: Off User: Off
Langauge intelligence Webex Assistant (WXA)	Webex Devices Webex App	Yes – But only when activated using the wake word, or when activated in a meeting.	Webex administrator (Organization and Site settings) and End User In-meeting controls	Site: Off User: Off
Language intelligence Real Time Translation	Webex devices Webex App	Yes – Used during a Webex Meeting	Webex administrator (Site setting) and End user In-meeting controls	Org: Off User: Off
Language intelligence Meeting transcript	Webex devices Webex App	Uses AI to convert meeting recordings into transcripts	Webex administrator (global setting for organization) and End User settings	Org: On User: On
Language intelligence Closed captioning	Webex devices Webex App	Yes – Used during a Webex Meeting	Webex administrator (global setting for organization) and End User In-meeting controls	Org: On User: Off
Room interpretation – People count	Webex devices	No	Webex administrator (global setting for organization)	Org: Off In call: Off Out of call: Off
People presence detection	Webex devices	No	Webex administrator (global setting for organization)	Org: Off
Ultrasound based pairing	Webex devices Webex App	No	Webex administrator (global setting for organization) Webex device settings Webex App: End User settings	Org: On Device: On Desktop app: On Mobile app: Off
WiFi based pairing	Webex devices Webex App (Desktop only)	No	Webex device settings Webex App (Desktop): End User settings	Org : On Device: On App: On

Webex and your privacy

Cisco considers privacy to be a fundamental human right, and we employ a responsible approach to developing AI. We are open and transparent about the personal data, including within user-generated content and metadata, that we collect and store in the Webex cloud when you use our services. Personal data protection and privacy considerations forms part of the design and development process for Cisco products and services. As such, all Cisco products and services undergo rigorous Privacy Impact Assessments prior to launch. For additional information on our personal information practices with respect to Cisco offers see the Webex App and Webex Meetings Privacy datasheets on Cisco’s Trust Portal.

More about Webex devices: https://hardware.webex.com/products

More about the Webex App: https://www.webex.com/all-new-webex.html