At a time when the accuracy of news sources is in question, and politicians want to restrict access to information, hyperlocal communications have more potential than ever. You Are Here is a project that essentially consists of a Wifi signal that broadcasts a single website. Users can connect to it on their cell phones to leave comments on a curated topic. The project was designed by a team of Tow Fellows including Sarah Grant, Susan McGregor, Benjamen Walker, Dan Phiffer, and Amelia Marzec. Two installations of the project occurred in Tompkins Square Park and the High Line. These are some observations on the challenges of attracting users to a hyperlocal project.

The public expectation of an open Wifi signal is simply that there is potential to connect to the Internet for free. If they find they aren’t able to complete their specific task, which may be to post to Instagram in the park, they will drop off and look for another option. This is the biggest hurdle for any Wifi-based project. You Are Here relies on messaging through posters at the sites of installation to instruct users on how to encounter the project. The audiences and needs in those locations–a barber and a restaurant–are different than the audiences that appear in the interviews for the curated content, which are parkgoers to Tompkins Square Park and the High Line.

The design of the system must fit the needs of the community. One of the requirements for success is to go into an existing community and find out what their specific needs are, rather than designing a system and then trying to build a community around it. This was a challenge with You Are Here. Hyperlocal communications would be most beneficial when there is a driving need for us to rely on our neighbors.

Designing the systems that carry information must be considered from every angle. A couple of the previous projects by the team members of You Are Here were designed as systems of protest: Occupy.here by Dan Phiffer, and Signal Strength by Amelia Marzec. Inherent in their design is the expectation for their use in a democratic setting, where users would set the tone of the conversations. These methods focus on the decentralization of information, and are less effective when used in a setting that requires top-down information.

Recently considered for the purposes of advertising, You Are Here and similar systems lack significant traffic that would validate them for use in a corporate setting. The failures of these systems as money-making machines contain the potential for them to be used for their original purposes—to give rise to citizen journalism and peer-to-peer communications.

For the long term, users need to care about the content and their interactions with each other. Within commenting systems, users will return when they are emotionally invested—even if that means they are having an argument online. Would an old-fashioned bulletin board be more effective? One benefit of that would be that people without access to smartphones would be able to participate.

In the event of a true emergency, with no connection to the internet, You Are Here could be a valuable addition to a specific local community, possibly if it were able to operate as a forum. Residents would be able to share how they got access to medical care and ATMs, or to discuss rumors from the outside. As we move into uncertain times regarding the use and delivery of information, we will need to be vigilant in finding the truth tellers in our communities that are focused on the greater good.

How can automation help reduce the cost of story discovery in a newsroom? As a Tow fellow in 2015, I had the opportunity to explore this question by building a tool, available online at campaign-finance.org, that automates some of the exploratory data analysis associated with investigative work.

Newsrooms need automated investigative tools now more than ever because their human investigative resources have been decimated. Since the 2008 crash, among the most severely cut departments have been the ones that do long-term, in-depth investigative reporting. Enterprise ideas are the hardest kinds of story ideas to come up with: you don’t know what you don’t know about what’s going wrong in our public institutions. Most investigative stories are reactive, meaning they result from a tip from a whistleblower, versus being proactive, resulting from the reporter’s own inquiry. Since newsrooms were cutting investigative resources, I wondered if we could build computational systems to help the remaining reporters do their jobs more easily and proactively discover story ideas in public data.

During my Tow fellowship, I explored whether a kind of custom software called a Story Discovery Engine could facilitate enterprise reporting on campaign finance issues. (It can.) I was curious about campaign finance in anticipation of the 2016 US presidential election. I had heard a lot about dark money and superPACs in the wake of the 2010 Citizens United decision, but I knew there was a vast amount I didn’t understand about this complex system. However, I did know that reporters who work with campaign finance data spend a lot of time doing the same routine things: downloading data from the FEC and other sources, cleaning it, associating data with known entities, and building basic visualizations. Any time you have routine processes, there is an opportunity for automation.

The last time I built a Story Discovery Engine, I optimized it to find a story in education data. This time, instead of starting with my own story idea, I started by interviewing other journalists. I specifically asked about the kinds of stories they look for, and what the indicators are that suggest a story might be hiding in the data.

To an outsider, these indicators are almost impossible to spot. But to these campaign finance gurus, the signs of corruption were clear as day. One common indicator is administrative overspending. Nonprofit organizations are required to report their income and spending. If the organization spends an unusually high percentage of its income on administration, it is often an indicator that something is amiss internally, and there is likely an opportunity for a story.

However, deciding what percentage is “unusually high” is a judgment call. It is also a judgment call to determine whether there is a story worth pursuing. Some fluctuation in administrative expenses is normal. There might be a perfectly good reason for an organization to have unusually high administrative expenses; a high percentage does not necessarily imply corruption. This ambiguity is the reason that it is unwise to build a system that claims to automatically identify investigative story ideas. It would be unfair (not to mention unethical) to accuse an organization or a public servant of misdeeds based on a naïve computational analysis. It requires human decision-making to fully consider what is going on in a given situation.

This human component is essential for newsrooms to remember. Computers can’t independently determine corruption. Currently, a human is necessary in any automated investigative system. Instead of a fully automated investigative system, in this case I built a human-in-the-loop investigative system. Most people dream of full automation: cars that drive themselves, robots that deliver packages. I don’t dream of this. I’m fine with a world that includes people. I like human judgment, as flawed as it is. I like the drama and the idiosyncrasies of human systems. The difference between a fully autonomous system and a human-in-the-loop system is like the difference between a drone and a jet pack. The drone is autonomous: it is programmed to go to a particular location, drop a bomb or take a picture, and then come back to base by itself. A jet pack (in theory) is designed to be strapped onto the back of a human being in order to accelerate the human’s effort. Both are legitimate system models, and each is useful for a different type of task.

Bailiwick, the system that I built, automates some of the grunt work associated with downloading FEC data, cleaning it, putting it into a database, organizing it into recognizable categories, and creating simple visualizations. The visualizations, plus the knowledge engineering layer that organizes the data into recognizable categories, allow a reporter to quickly make sense of complex data. Bailiwick analyzes data for each of the thousands of 2016 federal candidates and for 17,000+ active political committees. There is also an alerting function that allows a user to set up a personal profile of candidates or races to follow. A Pennsylvania reporter, for example, could choose to follow the two frontrunners in the PA Senate race and a handful of frontrunners in PA House races. Bailiwick sends an alert to the reporter via a private Slack channel every time there is a filing by a candidate the reporter follows. Setting automatic alerts, via Slack or a service like IFTT or Zapier, has long been known as an effective way to use automation in the newsroom.

Bailiwick is not small. As of January 1, 2017, the system included 11,032 lines of Python; 13,158 lines of HTML; 40,113 lines of JavaScript; 13,601 lines of CSS; 2.9 million lines of text; 2,892 lines of markdown; and approximately 94.2 million records in the database. The number of records increases every time new filings are added to the FEC site. The system gathers small updates from FEC every night, and then it completely refreshes all of its data from FEC once a week.

Bailiwick is not designed for beginners, but rather for experienced reporters who are specifically looking to find stories in campaign finance data. Top-tier news organizations like the New York Times or ProPublica or the Center for Public Integrity have software developers on staff who build custom, non-public-facing software for reporters to use on campaign finance data. Bailiwick is designed for organizations that don’t have internal developers.

For an investigative project, Bailiwick was relatively inexpensive to build. In Democracy’s Detectives: The Economics of Investigative Journalism, James T. Hamilton outlines some of the costs and benefits associated with complex investigative projects. He estimates that “Deadly Force,” the Washington Post’s 1999 Pulitzer Prize-winning story on D.C. police shootings, cost about $487,000 (in 2013 dollars) to create. Hamilton writes: “While accountability reporting can cost media outlets thousands of dollars, it generates millions in net benefits to society by changing public policy.” Watchdog reporting is good for society, but measuring its impact and its cost is not straightforward. The starting price varies by industry as well. Video is more expensive than print or digital: production costs for a single hour-long news documentary for a show like PBS Frontline start around $500,000 and can go upwards of $1 million.

A news app like Bailiwick starts at about $50,000 in up-front software development costs. It currently costs me about $1,000 a month to maintain; I plan to offer it for free online for about a year. Newsrooms are likely hesitant to commit to spending $50K on custom software projects. In the future, newsrooms could consider joining together to fund similar projects.

It is clear that news apps like Bailiwick can help newsrooms to produce more data-driven investigative work and can lower the cost of discovery for these investigative stories. Newsrooms who want to adopt this kind of automation will need to commit to up-front costs that are not insubstantial, and will need to plan for a development time frame that is longer than the customary time frame for daily or weekly news production. However, newsrooms excel at developing and meeting production timelines, so this challenge will likely not be an obstacle. It is also helpful to think about automated investigation as a kind of artisanal production. Data-driven enterprise stories are not easily mass-produced; they are artisanal products. Small-scale automation using human-in-the-loop systems is a cost-effective way to increase production of these high-quality stories, just as a small bakery might buy a large-capacity mixer in order to produce more loaves of bread. As newsrooms increase their production of watchdog reporting in the public interest, society will benefit.

Image via Sakena on Flickr.

In December 2016, the Committee to Protect Journalists counted 259 journalists imprisoned worldwide, more than at any time since the group began its annual imprisoned census in 1990. On January 25th and 26th, we hope you will join our effort to offer moral support and call for greater press freedom, by sharing the image below on social media along with the #FreeThePress hashtag. 

Journalists and others imprisoned for “crimes” like exercising free speech have said that communication from the outside world is a vital form of support. As a Chinese activist once put it:

“When letters and cards arrive at the prison like snowflakes, the prison knows that a lot of people are paying attention to these prisoners.”

With input from the Committee to Protect Journalists, the following group of organizations are holding postcard-writing and social media campaigns to highlight the situations of seven journalists. We hope you’ll join us by lending your support on social media.

ETHIOPIA

Ethiopian journalists work in some of Africa’s most restrictive conditions. Of the 16 journalists imprisoned there in December, two have spent a decade behind bars.

Darsema Sori, Radio Bilal
#DarsemaSori

Darsema and a colleague at the faith-based Radio Bilal were arrested in early 2015 and charged with 18 other defendants for allegedly inciting extremist ideology and planning to overthrow the government. Darsema is senior editor at the radio station, which extensively covered the Muslim community’s protests of a number of government actions – including the closing of Ethiopia’s only Muslim college. 

VENEZUELA

A year ago, there were no journalists imprisoned in the Americas. But this December, CPJ identified journalists jailed for their work in Cuba, Panama and Venezuela.

Braulio Jatar Alonso, Reporte Confidencial
@BraulioJatarA

Jatar was arrested in September a day after he wrote about protestors who greeted President Marduro with jeers and by banging pots and pans. Authorities claimed he was found with thousands of dollars in cash to be used for a “terror attack.” A legal rights lawyer charged that authorities planted the money and arrested Jatar, who is also a lawyer and political activist, for posting videos of the protest on his site.

CHINA

China has ranked among the world’s worst jailers of journalists for years; 38 were in prison in December, as Beijing deepened its crackdown on coverage of protests and human rights abuses.

Ilham Tohti, Uighurbiz
#IlhamTohti

Tohti is a Uighur scholar, writer and blogger who was arrested in early 2014. Authorities closed his website, which published articles on social issues in Chinese and Uighur. Tohti was sentenced to life in prison for promoting Uighur separatism on his website. He denied the charges, and his sentence was protested by the U.S. State Department, the European Union, and human rights organizations. 

INDIA

One of the riskiest areas for journalists in India is the Bastar region in the state of Chhattisgarh, epicenter of the conflict between Maoists and security forces. Pressures come from both sides in the conflict and are most severe for those reporting there full-time.

Santoch Yadav, freelance

Yadav, a freelancer in the Bastar region, reports for several local dailies, often on allegations of human rights abuses by police against tribal communities. In 2015 he was arrested and charged with rioting, criminal conspiracy, attempted murder, and supporting and aiding terrorist groups. His colleagues said the charges were fabrications brought in retaliation for Yadav’s human rights reporting.

TURKEY

Following the failed coup attempt of summer 2016, the Turkish government intensified its media crackdown, shutting more than 100 news outlets and arresting dozens of journalists. The 81 imprisoned in Turkey in December accounted for nearly a third of the global total of imprisoned journalists.

Musa Kart, Cumhuriyet
@MusaKart

Cartoonist Musa Kart was one of 12 staff and board members of Cumhuriyet, Turkey’s oldest newspaper, detained in October.  An official statement said they were suspected of producing propaganda for the Kurdistan Workers’ Party (PKK) and for what the government calls the Fethullah Gülen Terror Organization (FETÖ), two rival groups the government labels terrorist organizations. No date is set for Kart’s trial. 

IRAN

CPJ’s December census marked the first time in eight years that Iran was not among the five countries with the largest imprisoned journalist populations. But eight journalists remained jailed in December, two of them arrested in 2016.

Issa Saharkhiz, freelance
#IssaSaharkhiz

Saharkhiz has been imprisoned three times in seven years. His most recent arrest was in 2015, which led to his conviction for “insulting the Supreme Leader” and a three-year prison sentence, later reduced to 21 months. Saharkhiz is a prominent journalist who contributed to the opposition website Rooz Online. He still faces charges of “insulting the head of the judiciary.” 

EGYPT

Egypt had the third-largest population of imprisoned journalists – a total of 25 – in CPJ’s 2016 census, after Turkey and China.

Mahmoud Abou Zeid (Shawkan), freelance
#shawkan

Freelance photographer Abou Zeid was detained in 2013 while covering clashes between Egyptian security forces and supporters of ousted President Mohamed Morsi. After more than two years of pretrial detention, he was charged with weapons possession, illegal assembly, murder, and attempted murder. His trial was ongoing in late 2016, In November, he was honored in absentia with CPJ’s International Press Award.  

Note: This is the fourth post in the Beyond 140 characters series, which investigates how, why, and under what circumstances political journalists engage with Twitter. This piece shares some of the project’s key findings. The previous post reflected on how the nature of news events shape political journalists’ Twitter engagement.

Multi-platform and digital storytelling, understanding metrics and audiences, interactivity, and branding—these are just some of the issues facing journalists today, all while using tools and techniques that weren’t possible ten years ago, and with the added pressure of identifying a unique angle or scoop. If only some of these tasks are to be achieved and well executed in a given work shift, 24 hours in a day may not be enough. This is a theme—one of managing the many tasks and workflows—that has emerged in my interviews with 26 political journalists who work for some of the top legacy media organizations in the U.S.

Twitter has become an established journalistic resource, one that reporters are expected to constantly and actively utilize (as I discussed in a previous post on organizational pressures and institutional policies). Early skepticism appears to have morphed into conventional wisdom that, to use one journalist’s own words, “if you’re a journalist and not on Twitter, you’re a dinosaur.” But the story is not that simple. Rather, it is one of constant negotiation and evaluating opportunity costs.

Twitter as a distraction from core tasks

Among the journalists I spoke to, there appear to be two dominant, but opposing perceptions about Twitter’s place in everyday workflows.

The first narrative is one of Twitter as a distraction, where it competes for attention with traditional operations and production goals. Resources are limited, and the more space Twitter takes up in a journalist’s day, the less disposable time and energy remains for the items he or she needs to deliver. One journalist extensively talked about Twitter as a constant stream of overwhelming information and described his experience with the platform as follows:

It’s sort of like telling me that I should be watching 15 TV channels at the same time. I’m sorry, but I’m going to report and write and give you something of value, and I’m going to turn all that crap off that’s not going to matter to me.

Many others provided examples of distraction and information overload, which are taking a toll on productivity. Individuals with such experiences tended to engage with Twitter selectively and developed an increased awareness of the need to switch off. These journalists unanimously said they benefited from being more focused on their work. After all, as many highlighted, good writing is a craft and to do it well takes time and attention. One journalist explained his frustration:

We’re getting hit every week with a new tool or something else that’s cool. I got an email yesterday saying, ‘Hey, come learn Instagram.’ And then last week it was ‘Hey, come learn something else’. Are you kidding me?

On the flip side, minimizing distraction clearly involves opportunity costs. Many journalists are aware of this tradeoff, but have come to terms with it as the lesser of two evils. For example, not logging on at all or turning off push notifications can lead to missing breaking news or content shared by key individuals, especially politicians and their operatives, who are amongst the keenest adopters of Twitter as an immediate and interactive tool for public dialogue, outreach and mobilizing. These journalists risk falling a step behind colleagues and competitors, especially so when news might not be available elsewhere.

While news organizations may be willing to trade occasional Twitter abstinence for a better journalistic end product, things become trickier when journalists’ personal preferences immediately challenge their employer’s overall social media strategy. For example, how does management deal with those journalists who engage infrequently or inconsistently, when Twitter has long become an integral channel of content distribution for that news organization? Or how does one reconcile the incompatibility of the brand a journalist cultivates on Twitter with the core values and cultures of his or her employer? One journalist admitted:

I’ve kind of made my peace with it. It is what it is. If I don’t get a job because I don’t have a Twitter presence then, well, I don’t get a fucking job. That’s fine… But hire me or fire me for the stories, not Twitter or my tweets. I’m sure my bosses would like me to tweet more, but frankly, it’s just not a priority.

Twitter as a tool to optimize workflows

The second narrative is an opposing and more optimistic one, of Twitter as a tool to manage and optimize workflows. The factors that shape such perceptions aren’t dissimilar from those discussed above: journalists also referred to editorial and temporal pressures to produce content, for example, or an institutional drive towards branding or creating multi-platform visibility. But rather than considering Twitter a distraction from any one of such objectives, journalists view the platform as an extension of or an addition to the available means of achieving these. What sets those individuals apart is how they appropriate Twitter in such a way that it “fits into” the tasks and procedures they are already working on.

The vast majority of journalists I spoke to were generally hesitant to pinpoint concrete, realized benefits of their engagement with Twitter, largely due to the difficulty of assessing both short- and long-term outcomes. Yet, four distinct ways of perceiving Twitter in their day-to-day routines emerged, especially during intense news periods:

• Twitter as an early warning system, which immediately signals breaking news and alerts journalists of big stories. One journalist told me:
  

  I like the immediacy of Twitter and I like all the things that it’s designed to do. I can get the news that I want from various sources all at the same time, and I get it quickly. I feel like when something big is happening, I know it instantly – 20 minutes before you’re going to get the breaking news or an email from a news organization. It has become a constant in my life. It’s my favorite.

• Twitter as a digital notebook, which allows journalists to record (in a note-taking style, after all a tweet is only 140 characters) and immediately publish short snippets of information. Not only are journalists able to realize a competitive advantage with this approach in terms of being first to push out content. They are also later able to go back to their Twitter timeline and use its content as the backbone of the actual story they are tasked to produce. One journalist explained:
  

  I initially did not want to use Twitter at all. I thought I was just going to stay [at my news organization] as a writer on the web and for print.  But my editor at the time said we really need to adapt to social media. Use it as your notebook. And so I have really taken that to heart. A reporter’s notebook traditionally has always been where they keep their notes, their asides, observation, color, different scenes.  And they eventually call from their notebook and put it into a story. Twitter to me is groundbreaking for political reporters especially because you can now share your notebook with the whole world. If I know something is accurate, I’ve seen it with my own eyes, or I’ve heard them say it to me, I put it in my notebook and I have it there. Now I can also put it on Twitter. So it enables me to build a bigger story. Twitter I think is a component of your storytelling as a reporter.

• Twitter as a modern wire service, which already provides a curated selection of stories. This selection of stories is biased, of course, as it is determined by the users and conversations every journalist chooses to follow.
• Twitter as a direct line to political elites. Twitter tends to be driven by communities of interest, and the political community on Twitter is particularly strong. Many journalists described just how accessible politicians and their operatives are on the platform, when they are removed from their PR teams and personally share updates (as opposed to exclusively communicating via public statements or press releases). One journalist explained:
  

  I think for politicians, I’ve seen it especially in the past two or three years, that’s how they like to break news. They are getting a huge amount of political capital from Twitter – however hollow that might be – but they can certainly raise their profile from a single tweet. I think people see that; politicians see that. And it’s almost like a snowball effect. They see someone get some real mileage off a tweet, a series of tweets or an active Twitter presence, and they do the same.

Journalists also can’t be as easily shaken off, when tweeting at a politician is a publicly visible inquiry that demands a response, if only for the sake of accountability. One journalist says he has “definitely used Twitter to reach out to politicians,” but is also careful about the nature of their tweets, explaining that he “tend[s] to lose interest in those that just strictly use it as a PR tool.”

Despite all this talk around managing productivity and optimizing workflows, it was striking to find that only a handful of reporters use content management systems, such as TweetDeck or SocialFlow, that help to keep tabs on relevant topics and accounts of key individuals, especially those of political elites. Many admitted that they “probably should” be using a CMS. Others confessed their fatigue with the constant evolution of technology and scarcity of resources, leading to an unwillingness to learn how to use yet another (possibly short-lived) tool, however useful it might be.

Struggling to switch off

Journalists are naturally drawn to spaces rich with knowledge about current events, and Twitter’s “constant stream of information” makes the platform ever more appealing. News never stops and neither does Twitter. This makes it hard for heavily engaged journalists to switch off. One reporter described the following:

I mean we all feel this way, and it’s not really about journalism at this point, but all of social media – they’re apps, but they’re worlds. They’re platforms, but they’re actual worlds and communities. I think this creates a level of mental noise that might be unhealthy. There are always these conversations happening everywhere. I think the big challenge for me will be knowing when to unplug. So that when I wake up in the middle of the night I’m not reading Twitter.

Another journalist confessed:

I’ve become a little – I mean I hate to say addicted, but it’s just become much like we are. Our phones are always in our hands, or our pockets. I pick up my phone and I go through Twitter. In the middle of the night when I wake up, I go to Twitter. When I wake up in the morning I check Twitter. I find that it’s kind of become my primary news source.

The above accounts link to wider phenomena, such as concerns over work-life balance and the relationship between quantity and quality, as well as questions around the content and context of meaningful engagement. These insights are especially prominent in light of scarce resources and fierce competition in journalism and as concrete outcomes remain rather elusive beyond traditional journalistic practices and routines, both for those who heavily invest into the platform and those who engage selectively.

Image copyright Paul Strickland, used with permission.