Collaboration in the Age of AI: How Cisco is Pioneering the Use of AI and Emerging Technology Within Collaboration

Artificial intelligence (AI) has become all the rage. Just the mere mention of it makes us think of hi-tech and some futuristic state that promises simplicity and instant knowledge. According to research from O’Reilly, engagement with artificial intelligence technology grew 58% last year.* Additionally, the global artificial intelligence market is expected to grow nearly 50% in 2020, to a staggering $40.74 billion.** For the collaboration industry, their use brings hope of frictionless and instant connections.

Webex’s Rich History in AI Innovation

Cisco Webex has pioneered the use of AI within collaboration to bring this hope into reality. There is no other vendor in this space with a rich history in AI innovation like Webex. All for the purpose of helping people to connect like never before from wherever they work, play, or learn. To enable a safe return to the office. And to make decisions about collaboration spaces and office floor plans. From your home to the board room, our, AI, and automation technology provide intelligent experiences and drive changes, that keep everyone safe and productive.

And it isn’t something that will happen in the future. It’s happening today; in fact, we’ve been doing it for years following our strategy to apply AI and Machine Learning (ML) to practical applications in collaboration. We identified the most important areas where machine learning would make a difference in collaboration solutions and have focused our efforts on relationship intelligence, audio & speech technologies, bots & assistant, and computer vision.

This involved the application of a range of AI-based technologies including:

Conversational AI, a combination of natural language processing, dialog management, and question answering
Wakeword speech technology
Speech to text (STT) and text to speech (TTS)
Speech transcription and translation
Noise detection and removal
Face recognition
People insights

To support the advanced machine learning techniques used in these technologies several of them were optimized to run on NVIDIA GPU’s. Additionally, these features were deployed in the cloud or directly on client devices in order to provide optimal processing and the best data privacy position for end users.

2017

After several years of research and development, this was the year Webex first introduced AI that could change the way we meet and interact. For example, two big challenges we noticed back then were how we could reduce distracting noises (sirens, doorbells, dog barking, etc.), and how to present the best view of the conference room for remote participants. As a result, we introduced the following:

Machine learning-based noise detection – Webex used AI to recognize these loud and annoying noises in the background. Once detected, the system prompted you to mute your microphone or suppressed common noises such as typing on a keyboard or rustling papers.
Best overview and Speaker Track camera framing of participants – Video systems in the past were able to detect and zoom in on different speakers using multiple moving cameras. Modern Webex Rooms added intelligence to do this digitally with fixed cameras. They automatically framed up attendees as they talk to provide closeups of where the conversation is happening. This dramatically improved the experience of remote participants.

2018

The rise of small, agile teams meant that collaboration wasn’t just happening in traditional conference rooms. Every shared space was effectively turning into a huddle room. Whatever space people met in, they wanted the same easy join/start/share experience. So, our engineers not only made the conference room look even better to remote participants, but also improved the experience of people in those rooms. Webex Rooms systems have a modern hardware architecture that includes NVIDIA GPUs. This architecture allowed us to build sophisticated computer vision applications and bring AI-driven features to market faster. These included:

Face detection and people count – Webex Room devices used computer vision and a collection of sensors to determine how many people are in the room, unlocking powerful room utilization insights for customers
Presenter tracking – By detecting people and faces, Webex Room cameras could automatically follow the active speaker if they paced or moved about the room, so they always stayed in frame
Conversational AI on devices (Webex assistant) – Webex brought to market the first voice-activated assistant to help you call someone, start meetings or share your screen without touching anything
Automated pair and share: The Webex app connected to your Webex video devices wirelessly through ultrasound, and that’s when the magic happened. The proximity sensors in Webex Rooms could detect when you walked into a room, and the prompt on the screen would welcome you by name. And your Webex app could automatically pair to that device in order to share content without ever touching cables or fiddling with remote controls or cables!

2019

As the remote collaboration experience became better, what people wanted next was building a better, more intimate connection to the people they were meeting with on the screen. How could we shave off the 10 minutes of going around the room for introductions, and yet help you feel you already knew everyone you were meeting with?

Relationship intelligence (people insights) – Webex brought to market People Insights to provide users with comprehensive, real-time business and professional profiles of meeting participants, giving users context and increased insight about the people they meet with…either before the meeting or during the meeting.
Facial recognition with name labels – To go along with facial detection, we launched facial recognition. Adhering to strict data privacy rules, those who opted in for this feature were able to have the camera system recognize their face and then display their name label under their face to all remote participants.
Proactive collaboration assistant – With advancements in natural language abilities, Webex Assistant became even smarter. Previously, it was able to respond when spoken to and carry out actions. But now it could also proactively start a conversation. For example, when it was time for a meeting, Webex Assistant would wake up and ask the user if they want to join.

2020

This was/is the year where work from home become mainstream, and #RemoteWork started trending on social streams. Working from home presents some unique challenges and it became clear that new innovations were needed to make it easier to work from anywhere, without distraction. This has been a watershed moment for needing AI in collaboration, as more people dealt with all sorts of background and noise distractions. How did Cisco Webex respond? With intelligent technology for the hybrid workplace:

Background blur AND virtual replacement options across any device or OS – While other vendors offered this, Webex was the first to offer both blurring and virtual background across any device or operating system
Mask-friendly people counting: Webex Room devices are able to detect & count people regardless of which way they are facing, even if they are wearing masks! This information can now also be used for social distancing alerts based on room capacity.
Noise removal WITH speech enhancement – Solving for background noise has become table stakes. With the recent acquisition of BabbleLabs, Webex has taken the technology to reduce meeting interruptions to the next level. This noise removal technology, powered by AI, goes beyond noise suppression by 1) distinguishing speech from background noise, 2) removing background noise in real-time, and 3) enhancing your voice to elevate communication, independent of language.
A personal in-meeting assistant (expanding Webex Assistant in Meetings) – Now you have a personal collaboration assistant in every meeting! The Webex voice assistant expands beyond Room devices, to any Webex meeting, and uses advanced speech recognition and natural language understanding to turn talk into action.
Real-time closed captioning – See what is being said, even if you are in a place that makes it hard to hear what is being said.
Capture action items and highlights – Users can simply tell Webex to highlight certain points in a meeting or to create action items.
Searchable and editable meeting transcript – After the meeting see the transcript, edit it, search within it, and easily share it. It automatically captured for you.
Speaker labeling in transcripts– names are shown on notes, highlights, and transcripts to let you know who said what.

When working from home, many people are faced with spotty Wi-Fi or bandwidth constrained home networks that just aren’t as robust as corporate networks. We improved Webex performance in such conditions by applying machine intelligence in a few core areas:

Video super-resolution – When there isn’t enough bandwidth to deliver HD video, Webex intelligently applies adaptive super-resolution. We’re able to deliver HD-like quality even when receiving 360p or lower resolution video.
Region of interest encoding – Webex can intelligently identify the most important regions in a video frame, like a person’s face. When bandwidth is limited, Webex can still deliver high-quality video by making sure that the important parts of the frame look better, whereas other parts like backgrounds might be slightly lower quality.

AV1 Next-Gen Video Compression – AV1 is a new, next-gen video codec with an extensive toolset that delivers state-of-the-art compression performance. Last summer, in an industry-first demonstration we not only showed live encoding of 720p30 camera video at half the bandwidth of H.264 but also high frame rate share encoded at 1080p30 using around 2/3 of the bitrate of H.264 encoding 720p30, all on a commodity laptop. We’ve been making steady progress on this technology and soon you will see us implement it in Webex meetings, further reducing the amount of bandwidth required for a high-quality experience.

What About my Data Privacy?

Webex brings powerful artificial intelligence and machine learning to your collaboration experience, at home or at the office, to help to foster relationships, enhance customer interactions, and build high-performance teams across boundaries. But what about data privacy? How are my data privacy rights being protected?

Our AI/ML initiatives are guided by a few core principles:

Don’t retain data if you don’t have to
If you do, keep it for the shortest possible time
Be transparent about data usage
Provide edit and deletion controls
Empower end-users and admins

To get a more detailed view, I encourage you to read how Cisco treats the data it collects in this whitepaper. Additionally, this Privacy Data Sheet describes the processing of personal data (or personally identifiable information) by Cisco Webex Meetings.

Today and in the future, connected and intelligent workplaces (whether at home or at the office) will play a major role in delivering cognitive experiences that enable simple, frictionless, insightful collaboration. And we can’t wait to introduce more AI capabilities before the end of the year!