Microsoft and the Rise of Accessibility in Gaming

Gears 5 is the latest Microsoft title to gain accessibility features.
 

(Disclaimer before we get started. We are not affiliated in anyway to anyone mentioned in the article nor do we own any of the material linked (aside from the screenshot of Gears 4 which I took myself). We simply linked it for your convenience if you wish to watch it yourself.)

Microsoft is continuing it’s streak of making it’s games more accessible to everyone including those who are blind. In an interview with IGN (Click here to watch), design director Ryan Cleven talked about some of the accessibility options that will be featured in Gears 5, including how they’ve made it more accessible for those with visual impairments.

 Cleven had this to say about blind gamers and Gears.”We have people that play Gears that are actually blind, which I find is incredible. And we have, for example in Gears 4 we put in a system where you could, like a radar ping to tell where  the fortifications were, to be able as somebody who couldn’t see very well, to be able to find their way around the map. And like people could actually successfully complete horde while being blind is incredible and we wanted to embrace that sense of play, and make sure more people can enjoy Gears.” For those who want to skip to this point this all takes place at about 9:40.

What really bugs me about the interview is that as soon as Cleven starts talking about how they’ve made the game accessible to those who are blind the team at IGN decides now is the time to start playing B roll footage in the background (with sound on). Then before Cleven can talk about it further the interviewer then switches the topic to the B roll footage and how Terminator is in the game. I get that they probably had a very limited amount of time but I still found it off-putting and thought it worth mentioning. 

As both someone who is legally blind and a massive Gears fan the accessibility features have me really excited.  not only is the ping system making a return (which i didn’t use in Gears 4 but will most likely try out for Gears 5)  but now you can do things such as adjust subtitle size(will definitely be using that), change to a colorblind mode, text to speech for voice chat, and menu narration.  All in all this seems to be a massive step up from Gears 4’s accessibility menu which was limited to say the least.

This image has an empty alt attribute; its file name is gears-of-war-4-8_27_2019-1_04_49-pm-2-1.png

(screenshot that I took of Gears of War 4’s accessibility menu)

This isn’t the first time this year Microsoft has leaned into making gaming more accessible.  In may it was reported that Microsoft had patented a Braille accessory for the Xbox controller that would make gaming more accessible to those who are blind.

Thanks for reading, if you’d like to see us upload videos showcasing the accessibility features when the game comes out or if you have a topic you’d like to see us cover then please leave a comment.

Microsoft’s Seeing AI App Review

Note: this app was reviewed in tandem with two other scanner apps TapTapSee, and Vhista.

Seeing AI is a scanner app for the visually impaired developed by Microsoft for the IOS app store. The app uses an AI to narrate the world to mixed results.  The app includes nine scanner modes Short Text, Handwriting, Document, Product (bar-code), Person, Currency, Scene, Color, and Light scan.

First up is short text and handwriting. While it can recognize text semi-accurately it reads it in a random order making it an unusable mess. I tried scanning the cover of my copy of A Game Of Thrones because the text is in bold print which I thought would make it easier on the app. It ended up reading it as Martin Thrones R. R.

The handwriting mode works a little differently than short text mode. Instead of the text to speech continuously spewing word vomit, the app scans it into text which you can either read yourself if you have residual vision or have the text to speech read it to you if you’re full blind. I scanned handwriting three times and two out o the three times it was correct.

The document scan was a little more accurate than short text but was still pretty hit or miss. I scanned around three documents and only one was scanned with no typos. The other two times the words were either wrong or missing entirely. So if you need to have a document read to you I would look elsewhere.

The feature I found most useful was easily the product scanner. Each time I scanned a bar code it was able to correctly determine the product name. The only issue I could find with it is that people who are fully blind may have trouble scanning a bar code. The app is supposed to beep when a bar code is close but every time I scanned a product it failed to do so.

The person scan feature worked well enough. I scanned myself multiple times and each time the app got most of my facial details and expression right. My only issue with this feature is that it tries to predict age but ended up getting my age wrong (by about a decade) every time I scanned myself (about six times).  

The currency scan feature was surprisingly accurate, every time I scanned a bill it was able to accurately tell what the amount was. The only complaint I had was that it does not scan coins.

The biggest problem I had with Seeing AI was the general scan function or scene scan as it’s called in app . Almost every object I scanned was incorrect. So if you’re downloading Seeing AI for the general scan function don’t, there are much better apps out there such as TapTapSee.

Scene Scanner Accuracy Results:


Photo: Kingdom Hearts 3 PS4 Case

Object Subtitle: It seems to be screenshot, poster, cartoon, book, pc game, action-adventure game, action film, digital compositing, clothing, strategy video game

Accuracy: Wrong


Photo: Red Dead Redemption 2 PS4 Case

Object Subtitle: Probably a close up of a book

Accuracy: Wrong


Photo: Blade Runner 2049 Blu-ray Case

Object Subtitle: It seems to be poster, screenshot, vehicle, car, land vehicle, person, man, human face

Accuracy: Wrong


Photo: A Game Of Thrones By George R. R. Martin

Object Subtitle: Probably a close up of a sign

Accuracy: Wrong


Photo: Tazo Iced Green Tea Bottle

Object Subtitle: Probably an empty bottle sitting on a table

Accuracy:  Fairly Correct


Photo: Sun-Maid Organic Raisin Pack

Object Subtitle: It seems to be food, snack, dessert, baked goods, confectionery

Accuracy: Wrong


Photo: Red and White Cane for the Blind

Object Subtitle: it seems to be bicycle, bicycle wheel, indoor, toothbrush, stationary, writing implement, sports equipment, plastic, pen

Accuracy: Wrong


Photo: White, Grey, and Blue Converse Sneakers

Object Subtitle: Probably a group of shoes

Accuracy: Fairly Correct


Photo: Mead Five Star Notebook

Object Subtitle: Probably a close up of a computer

Accuracy: Wrong


Photo: White and Brown English Bulldog

Object Subtitle: a brown and white dog looking at the camera

Accuracy: Mostly Correct


Pros:

  • Free
  • Currency scanner
  • Product scanner
  • Quick processing time

Cons:

  • 3/10 accuracy for general image scanning
  • Money reader can’t scan coins
  • Face scanner can’t tell age
  • Inaccurate
  • Clunky UI (User Interface)

Conclusion: Overall Seeing AI was a mixed bag for me. For each good thing it felt like there were two caveats that came with it. However I still feel the app is worth a download for the product and currency features.

6/10

Thanks for reading, please keep an eye out for our review of Vhista as well as our round up of which of the three scanner apps is the best. If you have any suggestions of a review or topic you’d like to see us cover please feel free to leave a comment.

TapTapSee App Review

Note: this app was reviewed in tandem with two other object recognition apps (Seeing AI, Vhista).

TapTapSee is an Object Recognition app on the IOS app store, and Google Play store. The app is designed to help the blind and visually impaired by scanning objects through their phones camera.

TapTapSee’s user interface is by far the most user friendly out of the three object recognition apps I’ve been testing. Unlike the other object recognition apps there’s no category slider at the bottom to switch between what type of object you want to scan. All you do is tap the screen and it scans the image no matter what type of object it is , there’s no hassle of changing to a different category for certain objects.

The biggest draw back to TapTapSee is how long it takes to process an image once its been scanned. Once an image has been scanned it can take anywhere between 15 seconds and a minute to process the image.

Where it lacks in speed it makes up for in accuracy. almost every object I scanned was not only correct, but detailed. For example I scanned my dog and the app was able to tell not only the coloring but the breed (brown and white English Bulldog).

The app is also able to identify the titles of movies and video games based off of scanning the box, though it’s worth mentioning that it some times can struggle with more complicated box art. For example I scanned my copy of Kingdom Hearts 3 for PS4 and it identified as King Of Hears.


Photo: Kingdom Hearts 3 PS4 Case

Object Subtitle: King Of Hears PS4 Case

Accuracy: Wrong

This image has an empty alt attribute; its file name is kingdom-hears-3.png


Photo: Red Dead Redemption 2 PS4 Case

Object Subtitle: Sony PS4 Red Dead Redemption 2 Case

Accuracy: Mostly Correct

This image has an empty alt attribute; its file name is rdr2-tn-tts-4.png


Photo: Get Out Blu-ray Case

Object Subtitle: Get Out Blu-ray Case

Accuracy: Correct

This image has an empty alt attribute; its file name is gotn-2.png


Photo: A Game Of Thrones By George R. R. Martin

Object Subtitle: A Game Of Thrones By George R. R. Martin

Accuracy: Correct

This image has an empty alt attribute; its file name is got-tn-2.png


Photo: Tazo Iced Green Tea Bottle

Object Subtitle: Tazo Iced Green Tea Bottle

Accuracy: Correct

This image has an empty alt attribute; its file name is tazo-tn-2.png


Photo: Sun-Maid Organic Raisin Pack

Object Subtitle: Sun-Maid Organic Raisin Pack

Accuracy: Correct

This image has an empty alt attribute; its file name is rasin-tn-1.png


Photo: Red and White Cane for the Blind

Object Subtitle: White and Red Stick

Accuracy: Fairly Correct

This image has an empty alt attribute; its file name is cane-tn-1.png


Photo: White, Grey, and Blue Converse Sneakers

Object Subtitle: Grey Low Top Sneakers

Accuracy: Fairly Correct

This image has an empty alt attribute; its file name is shoe-tn-1.png


Photo: Mead Five Star Notebook

Object Subtitle: Mead Fire Star Notebook

Accuracy: Fairly Correct

This image has an empty alt attribute; its file name is notebook-tn-4.png


Photo: Adult White and Brown English Bulldog

Object Subtitle: Adult White and Tan English Bulldog (Original scan said white and brown but forgot to take photo, so had to try to replicate and this was the first scan. Sorry.)

Accuracy: Mostly Correct

This image has an empty alt attribute; its file name is daisy-tn-2.png


Pros:

  • Recognizes objects with surprising accuracy.
  •  Simple UI (User Interface)
  • Doesn’t require a subscription
  • Free
  • 9/10 Accuracy

Cons:

  • takes a minute to process
  • Needs voice over turned on to narrate

TapTapSee is easily the best object recognition app I’ve tested. While not the fastest it is by far the most accurate, with a 9/10 success ratio you’d be hard pressed to find a better recognition app.

Final Score:8/10

Thanks for reading, if you liked this post make sure to keep an eye out for our reviews of Vhista, and Seeing AI early next week.

CBS News Investigation Finds “Bad Braille” in Major Cities

According to an investigation by CBS bad Braille is being found in major facilities across the U.S.  Per the article there have been numerous complaints to the Justice Department’s Disability Rights division over bad or misleading Braille in Pennsylvania, New Mexico, Kansas, Illinois as well as several other states. What’s alarming about this is that the places lacking in Braille are hospitals, public libraries, and U.S. public transportation systems like the Albuquerque bus system.Most blind/visually impaired who live on their own have no other way to get around other than the U.S. public transit system, so to know that some of those bus stations either don’t have Braille or that the Braille is incorrect is worrisome to say the least.

The article also mentions that a branch of the D.C. Public Library is has a huge lack of braille on signs and no Braille labeling for audio books, which is slightly infuriating see as audio books are one of the primary sources of entertainment for the blind. Hopefully this gets addressed but by the looks of it the government has known about this for awhile and has done nothing about it so unfortunately this will likely stay as it is unless they get serious push-back from advocates. 

Thanks for reading, if you think there’s a topic we should cover, feel free to leave a comment. 

Sources cited: Dorsey, Steve. “Bad Braille Plagues Buildings across U.S., CBS News Radio Investigation Finds.” CBS News, CBS Interactive, 28 June 2019, http://www.cbsnews.com/news/bad-braille-plagues-buildings-across-u-s-cbs-news-radio-investigation-finds/.

Top 5 Most Useful Tools for the Visually Impaired.

Being blind or visually impaired can be pretty difficult, luckily there’s a variety of tools and devices that can make navigating the world a lot easier. Before we get started I’d like to mention that most (if not all) of the items on this list can be obtained through your local center for the blind if you are registered with them. With that out of the way here’s our list of the 5 most useful tools for the blind and visually impaired.

5. Talking Alarm Clocks/Talking Wrist Watches: It’s hard being able to tell the time when you can’t read a normal clock, luckily talking time pieces exist to make life a little easier. The alarm clocks are basically just a giant button that speaks the time and are reasonably priced at about $30. The wrist watches look like a normal wrist watch but the clock face is in large print, so this is a good buy for those who are fully blind as well as those who are legally blind.  The wrist watches are also reasonably priced between $20-$60. 

4. Large Print Keyboards: For those who are legally blind and don’t know how to touch type, this is probably something you should look into getting. They’re on the cheaper end of things at about $25, so if you do a lot of computer work this is probably worth the investment.

3. Kindle Fire: This may seem like an odd one, but for those who are going blind and still like to read, or just want an affordable tablet with excellent accessibility features then this is great option. The text size is highly adjustable so you can make the text as large as you need. The Kindle Fire has Audible pre-installed for those who are fully blind or just find Audible easier. The 10 inch Kindle Fire normally retails about $150 but frequently goes on sale for $120.

2. Magnifiers: If your vision is worse than a 20/200 (legally blind) then having a magnifier is a must. Non-electronic magnifiers are normally fairly cheap ranging from $13-$25, while electronic magnifiers that can digitally zoom in are a bit more on the pricey side at around $150. 

1.Canes: This one’s a no-brainer, but it’s an invaluable tool worth mentioning. If you are legally blind, night blind, or fully blind and don’t have one of these I highly suggest you get one. They’re pretty cheap at about $20 on amazon, or check with your local center for the blind as they normally provide canes to those who need them.

If you think there’s an item that should have been on this list please leave a comment. Tanks for reading.

10 IOS Apps for the Visually Impaired

Despite smartphones being a huge part in our everyday lives not a lot of people realize that there are a ton of helpful apps for the visually impaired. With that being said here’s our list of the 10 best IOS apps for the visually impaired.

 

Image result for vhista

10. Vhista

Seller: Juan David Cruz Serrano

Price: Free (In-App Purchases)

Stars: 5/5

What It Does: Vhista is an object Identification app that uses machine learning and your phones camera to narrate your surroundings. The app is also capable of describing other peoples appearances and can even predict emotions.

 

Image result for color inspector

9. Color Inspector

Seller: Aaron L’Heureux

Price: Free (In-App Purchases)

Stars: 3.8/5

What It Does: Color Inspector is a color identification app meant to help those who are color blind differentiate between different colors.

 

Image result for prizmo pro scanner

8. Prizmo Pro Scanner

Seller: Creaceed SPRL 

Price: $9.99 (In-App Purchases)

Stars: 3.6/5

What It Does: Prizmo is a document scanner app capable of scanning and recognizing text based documents, business cards, and images. The app uses voice guidance to help those who are visually impaired properly position their phone to scan the document. Once scanned the app is capable of recognizing the text and reading it back to you.

 

Image result for aipoly vision

7. AIpoly Vision

Seller: AIpoly Inc

Price: Free (In-App Purchases)

Stars: 3.6/5

What It Does: AIpoly is an object and color recognition app that identifies and narrates objects in the environment to those who are visually impaired. AIpoly can recognize over 1,000 items without the subscription and does not require an internet connection to function.

 

Image result for talkler

6. Talkler

Seller: Talkler Labs, LLC

Price: Free (In-App Purchases)

Stars: 4.5/5

What It Does: Talkler is a free app that lets you control your email via voice commands. The app can read your email to you, mark emails as unread, record a response, and delete emails.

 

Image result for taptapsee

5. TapTapSee

Seller: Cloudsight, Inc

Price: Free

Stars: 4.3/5

What It Does: Taptapsee is an object identification app where you tap anywhere on the screen to take a picture and then the app’s AI will recognize the object and speak the identification back to you.

 

Image result for be my eyes

4. Be My Eyes

Seller: S/I Be My Eyes

Price: Free

Stars: 4.8/5

What It Does: Be My Eyes connects those who are visually impaired and volunteers via video call, where the volunteer can provide assistance in completing simple tasks. 

 

Image result for nantmobile money reader

3. NantMobile Money Reader

Seller: IPPLEX Holdings Corporation 

Price: Free

Stars: 3.7/5

What It Does: Nant is a currency identification app that is capable of identifying 21 different currencies and then speaks the denomination back to the user. 

 

Image result for seeing ai

2. Seeing AI

Seller: Microsoft

Price: Free

Stars: 4.5/5

What It Does: Seeing AI is an identification app that utilizes AI to narrate the world around you. The app is also capable of recognizing text and people.

 

Image result for bard mobile

1. BARD

Seller: Library Of Congress

Price: Free

Stars: 3.6/5

What It Does: BARD is the national library service for the blind and physically handicapped, which provides over 100.000 free, downloadable books and magazines, as well as musical scores to those who qualify.

Microsoft Files Patent for Xbox Braille Controller

Microsoft has filed a patent for a Xbox One controller with a braille accessory on the back that would make it possible for those who are low vision or blind to enjoy gaming. The patent (shown below) appears to show a normal Xbox One controller with a device that hooks onto the back that can display in-game text and dialogue in braille via a touch-pad on the back. .The device is equipped with six paddles, (three on either side of the touch pad), this allows players to be able to read and react to things happening inside the game at the same time. The device also appears to have speech to braille capabilities which could prove useful when you receive messages from other players.

As someone who is going blind I’m ecstatic about this. I tend to spend a lot of my free time gaming which has become harder and harder to do as my sight goes. So hearing that there will be an easy way to at least play simpler, text based games makes me genuinely happy. There is the question of how many developers will support the controller and which games it will end up working with, as well as if it will only work with more text based games like the turn-based Final Fantasy games, or if it will work with faster paced games like first person shooters or fighting games.

Of course this is all just speculation since it’s only a patent, but it seems likely given Microsoft’s continued dedication to their accessibility controller they released last year.

An Introduction to Blind Resources

If you know of a resource that you would like to share with the community, please click here.