Dana Calacci cares about data analytics, but even more so, she cares about the people it impacts.
Or, as Calacci’s webpage states, “I study how data management and analytics impact community governance. Currently, I focus on how algorithmic management is changing the realities of work, how data management and participatory design can help create a new future of work, and the data rights of platform workers.”
In an article in the July issue of IEEE Spectrum, Calacci offered an example of a kind of cautionary tale that leads to an inspiring conclusion.
But along the way, Caracci also hinted at new ways of thinking about data, showing how some communities are pioneering new approaches to how they collect data, and even newer approaches to how they use that data.
Time for gig workers to unite
Caracci wrote that gig workers for the delivery app Shipt found that their “pay… had become less predictable” in early 2020. Previously, they were paid $5 per delivery plus 7.5% of the order value, but Shipt has since added new factors to its payment algorithm, such as mileage and time spent shopping.
“Shipt did not release detailed information about its algorithms, so they were essentially a black box that workers could not see inside,” they wrote.
But instead of accepting that, workers “came together to collect data and partner with researchers and organizations to understand the wage data,” Caracci wrote.
“It was clear that we needed to do something more, something more than just complaining in Facebook groups,” says Willy Solitz, a Shipt employee who organized the early efforts. “We needed to organize and do something,” Solitz recalls in a video from the MIT Media Lab.
Employees first took photos of their payroll receipts and collated the data, and by summer of that year, they had asked Calacci to build an SMS-based tool that could scale data collection more quickly: “By October 2020, we had received over 5,600 screenshots from over 200 employees,” they wrote.
Caracci wrote that the project was inspiring because it was “worker-led and driven by the measurement of workers’ own experiences. With the help of its tools, organized labor and their allies were able to effectively audit the algorithm and discover that it was giving 40 percent of workers huge pay cuts.”
But more importantly, “workers have shown that it is possible to create transparency in the face of corporate will and to push back against the opaque authority of algorithms.”
What the Shipt episode means
The new data sparked worker protests and media coverage, but Caracci’s article acknowledged that there was no answer as to whether it ultimately improved workers’ conditions: “We don’t know, and that’s a shame.”
In the discussion on Hacker News, some commenters argued that the data ultimately showed that more than half of Shipt employees did not experience a pay cut, but Calacci countered on social media that “the real problem is that pay has suddenly become opaque and unpredictable.”
So regardless of whether the new algorithms are fair, Caracci believes what matters is the power dynamic between workers and apps.
Or, as the article puts it, “In a more equitable world, this transparency would be available to workers by default.”
And ultimately, “our experiment has served as an example for other gig workers who want to use data to organize and raised awareness of the shortcomings of algorithmic control. What’s needed is a complete change in the platforms’ business models.”
I’m happy to share how an employee helped audit Shipt’s algorithmic payroll system. IEEE Spectrum ! Here are some discussions we’d like to cover on Hacker News (🧵): https://t.co/29Z7SERID2
— Dr. Dana Calacci (@dcalacci) July 10, 2024
And in an April article for the National Employment Law Project, Shipt employee Willie Solis wrote that “through crowdsourcing information in a Facebook group, we were able to prove that not all of our tips were getting to us as they should have.”
Other Community Source Data Projects
Caracci’s article cites a 2021 event showcasing other exciting worker-led data projects, reminding us that “there are researchers and technologists out there who are interested in applying their technical skills to such projects.”
In an email interview with The New Stack, Calacci also shared more examples of community-sourced data projects:
Cornell University has a Civics and Technology Lab (led by J. Nathan Mathias, assistant professor of communication) that was created specifically to work with communities to “study the social impact of technology” and test ideas for “transforming digital spaces to better serve the public good.”
In January, researchers at the institute, with the help of 155 undergraduate students, studied how hundreds of New York City employers responded to a law requiring them to test their hiring algorithms for bias. The results? “Of 391 employers, 18 employers published audit reports of their hiring algorithms, and 13 employers posted transparency notices informing job applicants of their rights,” the report states.
But the researchers concluded that “most employers implemented the law in a way that made it virtually impossible for job seekers to know about their rights,” and wrote that “the law gives employers extreme discretion in compliance and strong incentives to avoid transparency.”
Mozilla has an outreach/research team whose projects include a browser extension that lets you donate data about YouTube videos you’ve watched and regretted.
In 2022, data scientist Jesse McCloskey, along with Becca Ricks, head of open source research at the Mozilla Foundation, analyzed data provided by 22,722 people and concluded that “people feel they have little control over YouTube’s recommendations, and our research demonstrates that this is not the case.”
“We know that YouTube’s user controls influence recommendations, but the effect is small and most unwanted videos still get missed.”
The report ultimately recommended that YouTube should “provide researchers with access to better tools” and that policymakers should “enact and/or clarify laws that provide legal protections for research in the public interest.”
Caracci said the book “Data Feminism,” published in 2020, provides many more examples and promises “new ways of thinking about data science and data ethics that are informed by intersectional feminist ideas.”
LLM Hunger
It’s a topic that Caracci takes very seriously. “I would like to see more researchers treat community questions and concerns as a subject of serious investigation,” Caracci said in an email interview. “Many communities have real questions they want answered about their environment, the technologies they use, and how this is affecting their well-being and health.
“By treating the community as collaborators, as we did in the Shipt study, we can translate these into rigorous research questions.”
And Caracci believes this issue will only become more important in the future. Their article concludes that as worker monitoring tools become powered by AI, it could lead to even more inequality: “The battle gig workers are fighting is a front in a larger war for workplace rights that will affect us all.”
In an article published in the journal ACM Interactions last December, Caracci noted that we already live in a world with “algorithmically determined productivity scores, tracking of workplace text communications, and software that routinely takes screenshots of employees’ computer screens for employer review.”
” in [large language model] “In this day and age, such surveillance and the data it generates will be potential training material for future AI systems.”
But in a section titled “Starving the System as a Workforce Strategy,” Caracci noted a “sliver of hope” in the way the JD program requires human content in its training “for workers who want some control over the direction of future AI model development. If workers choose to abstain from work, they can effectively starve these systems of the data they need to improve.”
Automation continues to be touted as an unstoppable force, but this is far from reality. translation: Workers and their allies, researchers and advocates, can and should collectively push for automation in the workplace: https://t.co/iXwyZIbl82 pic.twitter.com/nzgGxBjrqm
— Dr. Dana Calacci (@dcalacci) November 16, 2023
In an email interview, Caracci outlined the larger fight: “Tech workers, along with creative workers, need to start considering how their data is being used to train the systems that govern other office workers and create new, valuable content. [generative AI]Their data helps determine what AI systems can do, which means gaining legal control over how that data is used is crucial.
“Workers should follow the example of the WGA and SAG-AFTRA and seek to negotiate data protection clauses in negotiations or form a union to gain the power to set data protection clauses,” they told The New Stack. “In the absence of federal or state protections for workers’ data, this is the best step workers can take to control and direct how technology will impact the workplace in the future.”
Caracci’s article highlights the potential of automation as an empowering tool and calls for the development of “participatory AI and governance.” However, he writes, “Achieving these goals will require more than just developing new technologies; it will require a paradigm shift in how we approach technology and automation in the workplace overall.”
“Instead of treating automation as an inevitable force to which workers must adapt, we need to recognize it as a social process that must be shaped by those most affected by automation: the workers themselves.”
Caracci will continue this work in his new position as assistant professor of human-centered AI research at Penn State. “To me, human-centered AI means first accounting for the impact new AI tools have on people,” Caracci said in an email.
But in addition, “it means building systems that mitigate those harms, from designing alternative algorithms and interfaces to deploying adversarial tools to resist or subvert existing AI.”
YOUTUBE.COM/THENEWSTACK
Technology is evolving quickly, so don’t miss an episode. Subscribe to our YouTube channel to stream all our podcasts, interviews, demos, and more.
subscribe
The group was created in Sketch.
David Cassel is a San Francisco Bay Area resident who has been covering technology news for over 20 years. Over the years, his work has appeared in a variety of outlets, including CNN, MSNBC, and the Wall Street Journal Interactive.
Learn more about David Cassell
