CAMBRIDGE — On any given Friday night at MIT there’s bound to be a few stragglers hanging around the Strata Center, the architecturally world-famous building in Cambridge that houses the MIT Computer and Artificial Intelligence Lab.
On Friday Feb. 23, there were more than a few folks about, and not just from MIT: people from Perkins School for the Blind, Amazon, the Museum of Science, Cisco, and even a representative from the White House gathered for Perkins’ “#HackDisability: AI for Accessibility Hackathon,” the Watertown-based school’s second-ever hackathon and the first focused specially on harnessing artificial intelligence for a group that is often left behind by technology.
The task was seemingly simple. A few dozen participants from MIT and surrounding colleges all volunteered to spend their week working in small teams and putting together new technologies to help people with disabilities better function in the world. Sponsored by Amazon, guided by mentors who live with a disability — many of them sight-related — and fueled by free catering from Wegman’s, the nine teams had from 6 p.m. on Friday until noon on Sunday to identify, design, build, and test their new products before presenting to a panel of judges for a chance to win $2,500 to further their research and development. Their products had to be focused specifically on using artificial intelligence
“Perkins has been around for 200 years,” said Dave Power, President and CEO at Perkins School for the Blind in Watertown, in his opening address. “And we want to make sure it’s around for another 200 years. And the way to do that is to make sure we’re paying attention to technology and the changing needs of those we serve. … (I)f we bring generative AI and people with disabilities to the table, some magical things are going to happen.”
Speaking to Watertown News, Power elaborated that even when new technology hits the market, it doesn’t always filter over to the community of people with disabilities. Or when it does, it’s just all wrong and unusable for a person with a disability. Power came to Perkins by way of venture capital, and said he’s seen a lot of well-meaning people invent things for disabled people without actually consulting the disabled people.
“People really don’t know what their problems are and what problems could be solved,” he said. “So the rest of us who don’t have a disability get cool stuff like VR headsets and watches. But people with disabilities can take a long time to get the benefit of new technology. So we want to say ‘yo, innovators, here’s a whole population with a different set of needs.”
At opening night on Friday, Judy Brewer, the assistant director for accessibility in the White House’s Office of Science and Technology rolled up in her motorized scooter and told the assembled guests and competitors “Without accessible technologies, it’s nearly impossible for people with disabilities to participate equitably in education, employment, health care, or entertainment. The way we develop technology is essential not only for security but also for public rights and safety.”
The star of the show was Joshua Miele, a 2021 MacArthur “Genius Grant” fellow and principal accessibility researcher in Amazon’s Lab126 — the branch of Amazon that made the Kindle and Echo, among other products. He’s also been blind since age 4, and made clear that he embraces the identity of being a blind scientist.
“Some statistics say 15 percent of the world is living with a disability,” Miele said. “And you might think that everyone’s dying to get rid of their white cane. I’ll tell you, as cool as the innovation challenges are, you will not invent something more reliable and that provides more information than a cane.”
So wherefore this Hackathon challenge, particularly with its focus on AI? The point, Miele said, was to train all these up and coming young engineers to start thinking with accessible design at the forefront of their minds from the get-go.
“These students are going to learn to make cool stuff with AI, but what they are also going to build is a better understanding and better knowledge of how to design for disabilities,” Miele told Watertown News. “The real outcome is not the tools they’re making. It’s helping them learn how to ask the right questions and build a better understanding of what accessibility means.”
As for the controversies swirling around AI, Miele said that these are all just more examples of bias in technology.
“If you design something from a standpoint that doesn’t really understand what the problem is, the tool is not going to be able to solve it,” he said.
Power was similarly sanguine about the possible issues of AI.
“We’re bringing together a community of people who can identify those problems,” he said. “AI has its shortcomings, but it’s coming. There’s no stopping it. So let’s embrace what it can do well and recognize the areas we want to be smart about.”
Replacing the Cane
By Sunday afternoon the remaining competitors in their teams looked a little droopier than they had on Friday, but they were nonetheless enthusiastic about what they had managed to build in 40 hours. Groups were hacking on everything from indoor wayfinding (because GPS doesn’t work inside), to emotional interpretation, to figuring out how to help blind people make infographics for their professional presentations.
Eden Adler, an MIT grad student on team VizAbility was one of those working on creating a way for blind people to make their own colorful, accessible infographics.
“Their work product, just like how mine is a reflection of me, it’s a reflection of them,” Adler said. “They don’t want bare minimum. They want it to look nice and have an eye-catching chart that excites people and makes the presentation pop.”
In terms of the AI question, Adler said, the tech has a lot of promise but there definitely need to be some guardrails.
“AI, and especially generative AI, has a lot of promise,” she said, “But also, taking humans out of the loop isn’t necessarily what these people need. They’ve expressed not wanting that.”
Over on the other end of the floor, team My Own Voice had created a video plugin to help better enhance communication between deaf people and hearing people on video calls.
“Right now it’s a broken pipeline,” said Aashini Shah, an MIT grad student on the team. “Deaf person have burdens of lip reading, which is only 20 percent accurate, or have to order an interpreter in advance, and there’s only one interpreter for every 50 people. Close captioning is imperfect, and there’s no emotion or tone or cadence in the words.”
The video plug-in, as designed by the team, would allow a deaf person to choose an automated voice that would speak for them for the duration of the video call. During the call, they can type their thoughts and mark up the texts with various emotions so they can be better understood.
Pauline Nuth, a doctoral student at the Harvard T.H. Chan School of Public Health, said that a big lesson learned was really forcing them to look at how they were designing things and making sure they’re accessible.
After an hour of presentations and judging, three $2,500 prizes were awarded — one of which went to My Own Voice, who said they were ecstatic with the win.
“We almost have to keep working on it now!” said Alexander Howard, a Microsoft employee on the team.
Miele said he thought the whole weekend was amazing.
“A lot of what I heard was people asking the right questions,” he said. “Asking what are the real problems, and how can we use technology to get the right answers. My hope is these kids will continue to think about these problem spaces and how they can improve the ideas they started with today.”
Powers was pleased with the results.
“This really was a success in every respect,” he said. “We have three really interesting solutions, and this event pulled together these students and these representatives to raise awareness in a way I think will be lasting.”