Group 2 Presentation Afro Lit

Afro Lit Group 2: Student Monitoring and its Unmonitored Harm

Omar Dukureh, Daniel Ramtulla, Faiza issa, Jordan Summers

Let's go!

What are academic Monitoring Systems?

Academic Monitoring Systems are technologies used by schools and universities to track student activity during exams. One example is Proctorio, which uses AI technology to record students while testing. Once the exam starts, the platform prevents students from switching tabs and limits their access to other websites. Proctorio's system works by flagging clips and sending them to the instructor to review for acadamic dishonesty.

Continue

Showcase of Proctorio

Continue

Percieved Positives of Proctorio

Proctorio's function is to automate invigilation through camera, input and screen tracking. The perceived positives of Proctorio are: - Less faculty will be required to supervise larger groups of examinees. - Cheaters will be easier to detect and won't be missed due to no eyes being on them at the moment. - Discourages cheating by acting as a "guarantee" that the perpetrator will be caught. Why these are perceived: - Proctorio records a person in their entire exam session and then flags the session if it suspects cheating. This forces the faculty member to watch EVERY SINGLE FLAGGED EXAMINEE SESSION. So rather than actually requiring less faculty and less effort it demands the same amount of attention and effort except all the burden can be placed on upon one person.

Continue

Long Term Negative Effects of Proctorio

• The use of AI monitoring systems such as Proctorio can trigger anxiety in students and worsen symptoms for those with chronic anxiety. In a survey of 597 medical students who used Proctorio, nearly 70% reported being afraid that the system would wrongly flag their exams.

• Discrimination towards marginalized students. Black students have reported being asked to shine more light on themselves because the system couldn't validate their identity, causing them to be denied access to the exam, while their white peers face no issues.

• A student researcher tested Proctorio's facial detection system using nearly 11,000 images from the FairFace dataset and found that the models fail to detect black faces 57% of the time.

• Students with children are often penalized by the system as Proctorio classifies background activity as a distraction.

• Proctrio requires access to webcams, microphones, and screen activity, which raises concerns about constant surveillance.

Continue

Connection to Unmasking AI Pt.1

A great title

“When companies require individuals to fit a narrow definition of acceptable behavior encoded into a machine learning model, they will reproduce harmful patterns of exclusion and suspicion.” ― Joy Buolamwini, Unmasking AI: My Mission to Protect What Is Human in a World of Machines

Buolamwini defines encoding as a topic similar to the “coded gaze”, but it is making prejudices a default setting for AI tools. In regards to functionality fallacy, it is seen as people having the assumption that AI is perfect, when there are often failures and mishaps when it comes to the technology.

Continue

Connection to Unmasking AI pt. 2

“AI will not solve poverty, because the conditions that lead to societies that pursue profit over people are not technical. AI will not solve discrimination, because the cultural patterns that say one group of people is better than another because of their gender, their skin color, the way they speak, their height, or their wealth are not technical. AI will not solve climate change, because the political and economic choices that exploit the earth’s resources are not technical matters. As tempting as it may be, we cannot use AI to sidestep the hard work of organizing society so that where you are born, the resources of your community, and the labels placed upon you are not the primary determinants of your destiny. We cannot use AI to sidestep conversations about patriarchy, white supremacy, ableism, or who holds power and who doesn’t.” ― Joy Buolamwini, Unmasking AI: My Mission to Protect What Is Human in a World of Machines

Proctorio has faced a ton of negative backlash regarding their facial recognition when it comes to students of color and with disabilities. Proctorio also has a bias of students with neurodivergent disabilities, students with high stress, and intersectional disparities. This stems from the white standard that the people who created AI are using and how minorities are labeled. AI is actively contributing to the prejudices that minorities continue to face.

Continue

Our Solution: The Apollo Sanction-Omar Dukureh

🤔

• A Sanction involves restrictive measures to enforce obediance with certain ideas

• Named in reference to Matter of Newby V Adelphi University (Orion suing Adelphi)

• Our Sanction explains where Proctorio goes wrong and changes we want to see made.

Apollo Sanction (Summary)

Due to poor language of terms of service, the company managed to avoid questioning and investigations/ decieve Educators

We demand for Proctorio to disclose internal information about their products and information omitted from the T.O.S to Administrators and parents/guardians alike.

We demand for a third party security team to further analyze the Proctorio software for any other details missed by our investigation

Final demand: Have Proctorio temporily pause their services so they can see what can be done about this brought up information

Continue

Conclusion, Questions, Comments, Thoughts, Concerns.

• Systems like Proctorio can discriminate due to not being optimally built to acommodate other skin tones or neurodivergence

• Just as Dr. Joy was able to point out the excoded members of different technology we did for proctoring systems.

• Hope for the future of proctoring systems.