Inside the Black Box of Predictive Travel Surveillance

In March 2020, Frank van der Linde entered the immigration line for European Union citizens at Amsterdam’s Schiphol international airport. Linde, a Dutch citizen and human rights advocate, was returning home from outside the EU, and the immigration officer asked him a series of questions about his trip. Linde thought it was a random check; after a few minutes, he was cleared for entry. But unbeknownst to Linde, his answers were recorded and shared with a Dutch public prosecutor, who was collecting information on Linde’s movements.

The officer had been tipped off about Linde’s arrival that day through a seemingly innocuous action that occurs whenever you board a flight to the United States, much of Europe, and increasingly anywhere in the world—the exchange of detailed personal data about each traveler between airlines and governments. The data, which is retained about you for years, is increasingly valuable for technology companies that are experimenting with using algorithms that could decide who is allowed to cross international borders.

Linde, who is publicly outspoken about homeless rights, anti-racism, and pacifism, was first secretly flagged by Dutch police in 2017 as a person of interest under an Amsterdam municipality counter-terrorism program. In July 2018, Linde had a “weird feeling” that he was being monitored; he would eventually sue the government over 250 times under freedom of information laws to uncover the extent of the surveillance. Although Linde was removed in 2019 from the city’s watchlist, later receiving a personal apology from the mayor of Amsterdam, the scrutiny continued. When Linde learned that the police had put his name on an international travel alert, he wondered if they were also using his travel data to track him.

In October 2022, Linde requested his flight records from the government. The data, called a Passenger Name Record (PNR), is a digital trail of information related to an airline ticket purchase. PNR records are sent by most commercial airlines to the destination country some 48 to 72 hours before departure. While PNR records might seem innocuous, they contain highly sensitive personal information, including the traveler’s address, cell phone number, date of flight booking, where the ticket was purchased, credit card and other payment information, billing address, baggage information, frequent flyer information, general remarks related to the passenger, date of intended travel, complete travel itinerary, names of accompanying travelers, travel agency information, historical changes to the ticket, and more.

In December 2022, over two years after Linde passed through Schiphol, the Dutch PNR office, called a Passenger Information Unit, handed over 17 travel records to Linde. They stated that they had not shared his data with others, but Linde was suspicious. He swiftly filed an appeal. In March 2023, the Dutch government admitted that in fact they had shared Linde’s PNR details three times with the border police, including ahead of the March 2020 flight, when the immigration officer was instructed to covertly extract information. (They also shared an additional seven flight records that they claimed to have only discovered on a second search.)

As Linde reviewed his PNR records, he was surprised to find that some of the travel data the government had on him was incorrect—some flights were missing, and in four cases, the government had records of flights he never took. For example, one PNR record from 2021 stated Linde traveled to Belfast, Northern Ireland; Linde says he had reserved the ticket, but changed his plans and never boarded the plane. “What do companies do with the data?” Linde asked as he scrolled through copies of the PNR records on his laptop. “If commercial companies help to analyze data that’s incorrect, you could draw all kinds of conclusions.”

In Europe, at least four technology companies—Idemia, SITA, Travizory, and WCC—offer governments around the world software that uses algorithms on traveler data to profile passengers. These companies claim their software can detect terrorists, human traffickers, drug dealers, serious criminals, missing persons and increasingly, people migrating without papers.

Products from these companies aim to combine multiple data streams about a traveler—such as your flight booking data with your visa application—to allow some people to pass quickly and effortlessly through border control. Those flagged by a machine as risky would be sorted into separate lines and subjected to a variety of measures ranging from questioning to physical searches and even possible surveillance by intelligence agencies. It would be difficult, if not impossible, in many countries to find out why you were flagged or what happens afterwards with your data.

In 2021, the Netherlands-based WCC claimed it was the first company to use artificial intelligence on travel data to conduct risk assessments. WCC’s Hermes software draws on PNR and Advanced Passenger Information (API)—the data of who has boarded the plane including name, nationality, passport number and immigration status—to use “predictive analytics to foresee previously unknown threats.” Others soon followed. Today, there is Idemia’s Traveler Analytics Suite; SITA’s API PNR Gateway, Advanced Passenger Processing, and Intelligence and Targeting software; and Travizory’s API-PNR Targeting System.

Idemia, SITA, Travizory, and WCC were advertising their travel surveillance systems, along with other products, at the World Border Congress in Skopje, North Macedonia in April 2023. Each year, a bevy of defense contractors, immigration authorities, technology ventures and security experts convene at the annual fair to discuss how to strengthen the globe’s political boundaries. In sessions on digital borders and new technology, Idemia’s sales rep talked about replacing passports with biometric face scans while Travizory’s front man advocated for “predictive border security” based on automated alerts and profiling.

For anyone who travels internationally, these surveillance systems may provide some convenience—but they can also flag you as a potential threat or even limit your freedom to travel, while giving you little ability to do anything about it.

Travizory co-CEO Renaud Irminger envisions a future in which borders will become invisible—for most people. “Everybody should be able to go out of his own country and into any country and come back without having to queue in line and being able to use only his face,” Irminger tells me enthusiastically over Zoom from his office on a sunny day in the Seychelles. “To make it happen, we need to connect the data from the traveler.”

Travizory, a Swiss-based startup launched in 2019, sells AI-driven border management software including API-PNR systems that assess travelers and gives them a color-coded risk rating (green, yellow, orange, or red). They also sell biometric exit and entry systems, hoping to combine the two products. “Our goal is to put a facial biometric corridor in [each] country, so people walk through the corridor, and either they are green and they can go to baggage or they are directed to see an immigration, custom, or health officer based on the risk which is identified,” Irminger says.

While Travizory’s system can clear someone to enter a country via machine, those on risk lists have to be further evaluated by a human being. When Travizory’s system flags a person, their name and other information gets sent to the relevant authority along with a tag like “low risk” or “high risk.” Different government agencies in a country—including customs, drug enforcement, immigration, and intelligence services—have different risk classifications, so the system can generate multiple risk lists. Travizory’s AI is trained separately for each country, and the data is stored on AWS, Amazon’s cloud or AWS Outposts in-country.

There are five ways that Travizory’s system assesses traveler “risk” using API, PNR, and other data streams. One is to try to match travelers against certain profiles or behaviors entered into the system by law enforcement or intelligence agencies. “Let’s say you have somebody in Kenya that is caught selling drugs,” Irminger says. The authorities could try to understand his background: “where he’s coming from, what computer he used, IP address, credit card payment, hotel; we look into what are the characteristics and [then] we can create rules. We have a tool where we say we flag this person as being like a drug dealer and the AI automatically will try to flag people who have similar attributes.”

The software also maps connections between known and unknown “persons of interest” and incorporates clustering algorithms to detect anomalies or “outliers” among travelers. Anyone who doesn’t fit a typical travel pattern, such as someone coming into the country for one day with multiple bags and staying at an expensive hotel, can be flagged in the system.

An outlier is not necessarily an indication of risk—as a journalist, my flight patterns might also look unusual—but Travizory believes that this unsupervised algorithm can eventually help law enforcement catch criminals who are trying to evade detection. “Preventing terrorist travel is fantastic, but to me the biggest gain would be preventing youth and women from being trafficked,” Irminger says.

I spoke separately with Morten Jorgensen, Travizory’s chief data scientist, who was in a hotel room in an unnamed African country when we connected over Zoom. Given van der Linde’s realization his travel data was incorrect, I wondered how Travizory’s profiling engines would account for similar inaccuracies. Jorgensen explained that there are gaps in the data, which becomes more complete the closer the plane is to take off. Travelers don’t usually have to enter a passport number, for example, until check-in. They could also enter their own data incorrectly. “When you build a risk engine like this, you have to put in logic to make sure that you work around issues when there’s gaps in the data and the data is always monitored,” Jorgensen says. Missing data triggers an IT alert: “There’s continuous work to make sure that the data is as complete as possible and correct as possible, but it’s never going to be 100 [percent] correct.”

I asked Jorgensen what variables went into selecting who looks unusual. “Everything we have on the passengers,” he replies, estimating that Travizory’s two AI engines use between 100 and 150 variables. “They’re kind of black boxes, so they will tell you that this person is potentially risky and this person kind of looks different, but how it makes this decision is kind of a mystery.”

Over the past few years, a global ecosystem of powerful governments, companies, United Nations institutions, and cross-border law enforcement agencies have pushed for the global exchange and, increasingly, AI analysis of traveler information. Passenger Name Records were created in the late 20th century so that travel agencies could link multiple legs of a person’s journey. In 2000, Australia began requiring airlines to send Advance Passenger Information for screening purposes, but the more widespread use of PNR and API travel data began after the 9/11 terrorist attacks, when the US government made it obligatory for airlines to transmit PNR and API records in advance of arrival. The EU followed in requesting API in 2004, and in 2016 began requiring flights entering the bloc to share the more data-heavy PNRs. In late 2024, the EU approved a new API directive that will require airlines to transmit passenger data to authorities before travelers reach the EU’s external borders.

What was initially a preoccupation of mostly Western governments changed as the UN and its less known organizations like the International Civil Aviation Organization (ICAO) began encouraging all UN member countries to collect, analyze, and share travel data. From 2014 to 2017, the United Nations Security Council passed multiple resolutions requiring the sharing of API and PNR. Initially, the UN focused on data sharing relevant to known terrorist suspects and specific terrorist groups, such as Al-Qaeda, but the remit soon grew to other possible terrorists and criminals.

Some governments built their own passenger tracking software and offered it for free to others. The US created the Automated Targeting System-Global system, which it has given to 24 countries, and the Dutch government developed the Travel Information Portal system, and donated the intellectual property rights to the United Nations in late 2018. Now called goTravel, the UN’s system is active in five countries.

Private companies were also quick to offer solutions so that countries without API-PNR systems—such as those in Africa, Asia, and the Middle East—could comply with the UN resolutions. Initially, most of the PNR/API software had similar features, including formatting the data and comparing traveler names to international watchlists or alerts. But soon companies began experimenting with new services. Travel data, according to a sales presentation given by SITA in 2016 and which is available online, allows for governments to understand “what people are doing, not just who they are” when they cross a border. This is despite the widespread flaws that can appear in the data—one government paper noted that PNR data can feature misspelled names, misplaced data elements, or abbreviations and therefore cannot immediately produce reliable intelligence. Emmanuel Wang, vice president for innovation at Idemia, told me when we spoke on Zoom that PNR data produced 48 hours before a flight is “completely decorative.”

Beyond counter-terrorism and organized crime, the companies selling API/ PNR software increasingly support integration with broader immigration enforcement services like visas and biometric border gates. Such is the case in Kenya, where in late 2023 Travizory sold its API-PNR system at the same time as it set-up an electronic travel authorization (ETA) system, replacing Kenya’s previous e-visa system. When the ETA initially launched in early 2024, it required citizens from countries that previously could enter without a visa—such as Ghana, South Africa, and Malawi—to apply for and pay for the authorization. The fee is $32.50 per person, and Travizory takes a percentage. A backlash from African countries led to the Kenyan government suspending the fee for some countries, but citizens still have to fill out the online form, which generates an additional data trail, such as hotel information and in some cases bank statements. (The Kenyan government retains this data.)

Travizory’s footprint is still relatively small, though Irminger says they are a leading API-PNR system on the market. Travizory’s system is live in the Seychelles, Kenya, and two undisclosed countries. Only eight African nations currently have PNR systems, something Travizory hopes to change: the company is in discussions for deployment in 20 other African countries.

SITA is the opposite of a startup like Travizory: It’s a multinational that claims, “almost every airline and airport in the world does business with SITA, and nearly every passenger trip relies on SITA advanced technology.” SITA, founded in 1949 by airline companies, offers airline communication infrastructure and services like baggage tracing. (Irminger and Jorgensen used to work for SITA). According to corporate sales materials and an emailed statement, some 75 governments use its border control services like electronic travel authorizations and API/PNR data exchange In addition, SITA’s Advance Passenger Processing (APP) software moves away from using just API/PNR data to creating an “Expected Movement Record” for each passenger. The Expected Movement Record includes “interactive API” data, known as iAPI, which is collected by SITA’s software when the person checks-in and sent to all governments along the person’s journey. According to SITA, “Participating governments are then able to respond to the carrier … in real-time, authorizing or denying the boarding of each and every passenger.”

In June 2023, SITA launched its AI-driven Intelligence and Targeting product. The software utilizes a person’s Expected Movement Record and other data sources—such as visa data, profiles, in-country events, and API/PNR data—to issue automatic risk assessment scores for each traveler. A score over a certain threshold generates an alert. To improve its algorithms, SITA’s model can be updated with information such as interview transcripts, photos, and detailed reporting from border police and other officials. As with other companies systems’, the purchasing government has full control over the data collected. Some 431 million travelers are “risk assessed” annually via the Intelligence and Targeting system, which is configurable for use by multiple government agencies such as border control and the police. A SITA brochure states the product can “discover further threats” and save all system searches for “matching against future events.”

I asked SITA, whose interactive API software is used by the United Arab Emirates and Qatar, if they had any mechanisms in place to prevent governments from abusing this system. For example, what would happen if a government wanted to prevent a dissident politician or human rights activist like Frank van der Linde from boarding a plane? A corporate spokesperson replied in an emailed statement that SITA could not comment “due to the confidentiality commitments it has made in the relevant contracts,” , but according to a position paper accessible on their website, the company’s “risk assessment capability enables a government to export its border to every single point on the globe where passengers can board flights, ships or trains bound for their territory.”

Inside the EU where these four companies are based, there are privacy protections around citizens’ personal data through the General Data Protection Regulation (GDPR) framework.

Under GDPR, EU residents can request their travel data as Linde did. Multiple court challenges in Europe have clamped down on the length of PNR data retention by governments, and in 2022, the EU’s Court of Justice largely banned automated risk assessments of travelers based on PNR data, citing the potential for human rights abuses. As a result, WCC wrote in an email statement that their software “ensures all decisions involve human intervention.”

But governments can claim exemptions under GDPR for “national security.” Idemia’s Traveler Analytics Suite, which mentions it is GDPR compliant, is currently in use in France. (The EU’s AI Act, which passed over the past 24 months, also has a national security exemption.) “With IDEMIA TAS, French law enforcement agents now have the advanced, automated crime-fighting tool they need to detect risks and threat patterns in real time from a huge and growing volume of passenger data,” proclaims Idemia’s brochure.

When I spoke with Wang, he said that Idemia’s algorithms are for screening purposes like reconciling misspellings or different versions of the same name, and targeting, which allows the French government to set-up its own watchlist. No risk scores are given, and PIU staff check automated matches against the watchlist. For five years though, the system retains if there was an alert on a passenger. Wang specified the content of the alert is not saved.

European-based companies can also sell their software to non-EU governments, where in the absence of privacy regulations traveler data could be indefinitely stored by a government. SITA mentions in its sales materials that how quickly its system can be set up in a country depends on the legal context: “We have first-hand experience working in countries where regulations needed updating to provide a clear legal basis for both transport operators and passengers when introducing new ways of operating controls at the border.”

Irminger tells me that Travizory has a clause in its contracts which bans countries from using its system to detect sensitive passenger information such as race or religion. Airlines typically mask out other sensitive data, such as meal requests, when they send PNRs to governments. Travizory recommends that countries follow ICAO guidelines, which suggest deleting PNR after no longer than five years. But the company says it cannot monitor whether its clients adhere to these guidelines.

“We have no idea if these systems are accurate, the extent of the data they’re collecting, or the human harm,” alleges Anna Bacciarelli, a senior researcher in the Technology, Rights and Investigations Division at Human Rights Watch. “The potential for harm here is absolutely massive.” None of the companies selling predictive services offer information publicly about passenger redress if they are unfairly targeted by the algorithms or have posted openly accessible human rights or privacy impact assessments. Idemia, Travizory and SITA told WIRED that they do conduct privacy impact assessments, but that these are confidential. According to Travizory, “the government is ultimately responsible for human rights and or privacy impact assessments” while SITA claims governments and airlines impose on them “confidentiality obligations.”

“The fact that it’s a black box is extremely worrying, because there’s no real way of saying X person should definitely be on that register and here’s how we reached the decision,” says Bacciarelli. At least one algorithm experiment on immigration control—the UK’s program to fast-track certain visa applicants—was halted after advocates claimed the streaming algorithm was racist. The use of machine-based systems to deny boarding to passengers “could potentially undermine the right to seek asylum,” Bacciarelli adds, noting that air travel is one of the ways that refugees arrive in a new country. Concerns about the use of AI-driven software to deny boarding to passengers was also flagged in a scathing report issued by then-UN Special Rapporteur on the Promotion and Protection of Human Rights and Fundamental Freedoms while Countering Terrorism, Fionnuala Ní Aoláin, in December 2023. The report, which reviewed the UN’s own goTravel system, alleged that it represented “a profound human rights risk and a serious reputational risk for the UN itself” and should be immediately paused. (It remains active).

Even the claim that the machines will better detect human traffickers raises alarm bells for Bacciarelli, who highlighted the possibility of automation bias. For example, air stewards are trained to spot signs of human trafficking; if they now rely on machines to flag potential traffickers, there’s a chance criminals could go undetected because they were given a green light by a machine.

Security experts are also skeptical that the software can deliver on some of its promises. “If the intelligence community or law enforcement is already interested in a person, I think the AI targeting would be helpful, because you can find things in the system that you may have overlooked,” says John Harrison, an associate professor of counter terrorism at Rabdan University in Abu Dhabi. “But I don’t know that you could necessarily predict that somebody is a terrorist or narcotics smuggler simply based on the fact that some algorithm says that these travel patterns dictate that.”

Despite the risks posed by algorithm-driven profiling, the security sector is moving quickly to develop and install similar systems for every mode of transport: cars, buses, trains, and ships. Companies are following the lead of many of Europe’s governments. In a closed EU organized meeting on innovative technologies for border control in July 2023 in Warsaw, Poland, materials obtained under a freedom of information request reveal that the Dutch government referred to plans to scale up travel data exchange and targeting at borders in a powerpoint as a national “surveillance system [to] process passenger data to combat irregular residence or stay, linked to irregular migration,” while the Belgium government offers ideas for extending PNR to rail and bus lines.

SITA now envisions a “multi-modal borders system” and advertises that its designers are working to integrate tracking passenger information sharing for planes, ferries, road and rail into the API PNR Gateway. A 2022 op-ed that ran in the Border Security Report by Manu Niinioja from WCC, advocated for more “national targeting centres” that track all transport modes and combine traveler booking data with visa applications and facial recognition at the port of entry. (The UK, the US, and others already have national targeting centers that centralize traveler and transport information.) The challenge of combining this data can be solved by “advanced commercially available software solutions,” writes Ninioja. For these companies, more opportunities to track where people are moving results in higher profits. For the traveler, they run the risk of being ensnared in an increasingly complex web of surveillance.

In mid-September, I met Linde for coffee in Berlin, where he told me that trying to unravel the real-world consequences of his watchlisting had become his life’s work. “I don’t know if I’m still actively surveilled,” Linde says. “I’m not surprised by anything anymore.”

This story was produced by Lighthouse Reports, Europe’s investigative newsroom.