Satellite and aerial surveillance for migration: a tech primer

What this means is that the closer a satellite is to earth, the faster it must travel to compensate for earth's gravity. Satellites in Low Earth Orbit go much faster than satellites that are in higher altitude, meaning they revolve around earth multiple times a day. This is why some satellites can get information from the same point on earth more often than others. It also explains why you can get satellite TV by pointing your antenna toward a specific direction: TV satellites are indeed geostationary, (37,000 km above earth) meaning they behave like a fixed point in the sky, essentially moving at the same "speed" as us and allowing us to get our favourite TV show on.

Logically, a satellite which revolves around earth multiple times a day has the capacity to take more images of a particular location, which has relevance to monitoring boats. However, as we explore later, monitoring objects at sea is a complex matter.

Check out this interactive graph showing how satellites in geostationary orbit behave like a fixed point in the sky and how satellites in Low Earth Orbit can rotate around earth many times a day: https://upload.wikimedia.org/wikipedia/commons/b/b4/Comparison_satellite_navigation_orbits.svg

As you can see, the point in the geostationary circle (the largest one) is always above the same part of the earth and takes more than 20 hours to complete a full rotation. On the other hand, the International Space Station (ISS), that is in Low Earth orbit, moves much faster and only takes a few hours to complete its orbit, meaning it gets to observe much more of earth's surface in one day.

When it comes to the tech they use, satellites can be equipped with a multitude of sensors, radars, antennas and cameras to achieve different purposes. Google Maps/Earth for example is made (in part) with images from the Landsat program, a set of NASA/USGS satellites which include a super advanced camera and a large range of sensors. These satellites are also in Low Earth Orbit as they try to capture changes at a relatively high frequency.

Other satellites such as the ones used for GPS are equipped with radio transmitters allowing people to geolocate themselves by receiving signals from 4 different satellites and doing some maths. More invasive satellites such as the US NROL-44 are equipped with massive antennas and sensors for military purposes, enabling it, for example, to intercept and interfere with communications.

Aerial surveillance and migration flows

Within the more than 3,000 active satellites orbiting earth, some are dedicated to monitoring migration flows or used for this purpose. Those are either owned by governments and agencies like Frontex or by private companies that then sell access to the data they collect.

Given the precision of the sensors they can embed, satellites are often limited by the size of the object they monitor e.g. they can only monitor the movement of object of a certain size. In the context of migration, this means a focus on monitoring boats movements or camps' size.

When it comes to aerial surveillance, satellites are not the only option. Drones are used, for example, to monitor migration routes, on land or sea. They can be controlled from a Ground Station and via satellites.

Let's take a closer look at satellite imagery and UAV/drone surveillance:

Satellite imagery

In the migration context, satellites equipped with cameras, optical sensors or active sensors can be used to monitor refugee camp size and the movement of large object such as boats. To achieve this, most of these satellites can be found in Low Earth Orbit as it is closer to the earth and enables them to capture multiple images of an area every day.

Use of satellite imagery for this purpose has been around for more than a decade, with examples reported by Amnesty International in 2007 of the Spanish authorities presenting satellite photos to the Mauritanian authorities to demonstrate that migrants departed Mauritanian territory. More recently, in 2015, Frontex released a copy of a satellite image (wayback machine link to the original publication) used to detect 370 people on inflatable boats off the Libyan coast.

Camp size monitoring via satellite is used by both border control agencies and NGOs and humanitarian organisations to illustrate their work or adapt their actions.

Examples include MSF using the EO4Hum project to scale up its activities in northern Cameroon, WatchTheMed, an NGO using satellite data (among other things) to monitor and document the deaths and violations of migrants’ rights at the maritime borders of the EU and UNITAR using satellites to map refugee camps in Jordan.

Satellite imagery quality varies greatly depending on the equipment used. As a rule of thumb, the better the quality is, the smaller the zone surveilled is likely to be. The choice is between having images of very large regions with a low definition (only object over a certain size, dozen meters, are detectable) or images of much smaller areas with a higher definition.

The reason is satellites need to be at nadir (directly above the point they are observing) to avoid missing elements hidden because of the angle at which the image is captured (imagine taking a picture of a 100-storey from above, if you move in any direction the building will hide part of its surrounding). This means the camera movements must be limited to avoid this side effect. This also means that you need to be further away to capture larger areas with limited distortion and minimum hidden objects (images of the entire globe are usually taken from geostationary satellites).

Additionally, given the very high resolution of the captured images (tens thousands of pixels) and the other metadata collected (geolocation, additional frequencies such as infrared), satellites might be limited by their on-board computational power (much in the same way that your camera or smartphone takes time to process a photo after you've taken it) limiting how many images can be captured in a given time.

Depending on the sensors, satellite imagery can also suffer from the weather and be incapable of taking pictures because of clouds. This can be avoided with active sensors that emits a beam of radiation, allowing satellites to "see" through clouds.

UAV/drone surveillance

Drones and Unmanned Aerial Vehicles (UAV) are a growing market and have been deployed in more and more situations over the past few years, from military intervention to reconnaissance, raising wide concerns regarding their usage. These concerns have been fed by civilian casualties caused by US drone strikes and the use of drones and UAVs is widely discussed by international organisations (it's a recurring topic at the UN that's been the focus of many reports).

While drone and UAVs can have widely different form factors (sometimes as small as a hand), the devices used for migration surveillance purposes are usually much bigger and heavier than commercial or recreational drones, easily reaching the size of a bus. The Elbit Hermes is 12m length, 15m width and weighs 1,180kg for example. To navigate, they are either autonomous, using sensors and input to follow a set of instruction, or are remotely controlled by a human operator. They can, for example, be fed GPS coordinates and monitor specific areas for a given amount of time, as well as take actions in certain circumstances (e.g.: if an object of interest is spotted).

An autonomous drone will usually have a mechanism to be controlled manually if needed. Some drones have up to 50 hours of endurance like the Heron 1, meaning they can patrol areas for long periods of time. These devices can be controlled by ground based, direct line-of-sight (LOS) data link systems (meaning a ground station transmit instructions if the UAV is in line of sight) or via satellite communication (SATCOM) should the drone be out of reach.

Data sharing is usually two-way and Drones/UAV can transmit back information to ground stations such as images, detected object or any other message. They can be equipped with a wide range of sensors from cameras to thermographic cameras, to radar systems enabling them to detect targets in a variety of environments.

There are many examples of drones deployment for border control and surveillance of migration flow. In the UK, a £1 million contract was awarded in 2020 to an Israeli defence company (Elbit Systems) to demonstrate and develop drones to enhance coastal surveillance operations (contract). The call for tender sought a company to “assess the potential use of UAV to augment current and future aerial surveillance capability by reducing, enhancing or replacing existing delivery methods.”

This builds on the existing use of drones to monitor the English Channel which appears to have been first revealed in December 2019. In this instance, the BBC revealed that a single unmanned aircraft (an AR5, manufactured by Tekever) had been spotted flying from Lydd Airport to monitor people attempting to cross the channel from France by boat. This 7.3m by 4m drone is a device specialised in near shore and maritime operations and is optimised for search and rescue missions, detection of oil spillage and anti-piracy and traffic control missions. It embeds a range of high precision video, imagery and sensor and offers Beyond Line Of Sight (BLoS) satellite communications (ie.: it can be controlled and communicate via satellite when out of range of the ground station). The European Space Agency settled on the same drone for its project pollution monitoring and search & rescue project RAPSODY.

Similarly, Frontex, the European Border and Coast Guard Agency, has reportedly used drones in the Mediterranean to both detect migrants vessel and harass search-and-rescue charity boats. The Guardian has found three contracts for these drones, totalling £95m. The drones used include the Hermes, made by Elbit Systems, and the Heron, produced by Israel Aerospace Industries. These drones have been reportedly deployed shortly after the end of EU’s main naval mission in the area, Operation Sophia, replacing boats obliged under international and EU law to assist vessels in distress with unmanned vehicles that are not obliged to be equipped with life-saving equipment. In 2020, the agency also awarded a €100m contract to Airbus and two Israeli arms firm to operate UAVs to spot refugees and migrants attempting to cross the Mediterranean sea to Europe, according to EU contracts. A drone bought in result of this contract is reportedly deployed in Malta where Frontex will be operating it. No details have been disclosed about the technology it will carry but the tender mentioned at minimum "Thermal imager; daylight zoom camera; daylight spotter; giro –stabilized turret" a.k.a a stabilised cameras capable of seeing in day and night light, "Maritime Surveillance Radar" and optionally "Satellite phone location equipment; GSM phone location equipment;".

Operating UAVs can have widely different operational costs. These costs aren't publicly available for the examples described above and it is difficult to find any detailed figures given that these costs depends on the organisation deploying the drone and the mission for which it is being used. An example of operational costs for operating a drone in a military context can be found in the Institute of Defense Analyses' paper on Staffing for Unmanned Aircraft Systems (UAS) Operations which assesses the costs of the Department of Defense's (DoD) UAS operations. This paper details how these operations can require between 2 and 39 people to function and expose the $19M DoD buget to operate the entirety of their fleet. While these costs include tasks and qualified staff specific to the military context, it can give an approximate idea of the requirements to operate UAVs operation.

Of note: GPS and satellite phone interception

Since we are on the topic of satellite, it's worth talking about satellite phones. While this technology isn't designed for surveillance purpose, it can still be exploited as such given its technical nature.

Satellite phones are, as the name indicated, communication devices that rely on satellite. As such, they can be used from almost anywhere on the surface of the planet (the fewer objects between you and the sky the better), without relying on the terrestrial antennas network. With this in mind, it is technically possible to intercept satellite phone communications. This could be used to intercept smugglers communications as satellite phones can be the only reliable communication device in maritime areas. Horizon Technologies, for example, claims its FlyingFish SIGINT system can intercept metadata and (when combined with demodulating tech) content from satellite phone conversations.

Data sharing

The use of satellite imagery and UAV drone surveillance is not only carried out by those who act upon this information but is also part of data sharing practices. For example, it is reported that Frontex uses this to inform the Libyan coast guard, rather than acting themselves. This has been condemned as an abrogation of ‘responsibility to save lives.’

Innovation: coming to a satellite near you?

Migration is complex. The number of people arriving via small boat for example in the UK is reportedly a small fraction of arrivals. It is a precarious and dangerous journey.

Satellites are being deployed and used as part of the aerial surveillance toolset despite these potentially being expensive to deploy and suffering limitations such as the need to be targeted at specific areas and only able to spot objects of a certain size.

Yet the on-going wave of technological improvements is transforming how satellites are used and what they are capable of. Let's have a look at two of these technological innovations:

CubeSat and rocket innovation

With a design proposed in 1999 by Standford University professors, the CubeSat is a type of miniaturised satellite made of multiple cubic modules of 10cm x 10cm x 10cm. Cheap to produce and usually quite light, these satellites can be loaded as secondary payloads on rockets (i.e.: they are not the main reason why a rocket is being send to space but they take advantage of the available space to be deployed), making it fairly easy to launch multiple satellites for a moderate cost (secondary payloads are cheaper to launch because of the limited control offered regarding time and date of launch).

While the design isn't recent and most CubeSats were launched by academia until 2013, this type of satellite has seen a sudden surge in interest with the development of commercial space transportation companies, like SpaceX or Rocket Lab, as well as the advances in rocket re-use. Combined, these two factors enabled the deployment of multiple satellites more easily and at a much lower cost than traditional devices ($100,000 instead of $500m for a "traditional" satellite), with some missions launching more than a hundred CubeSats at once.

This mini-revolution has opened the door to any type of earth observation project, making failure acceptable given its relatively low cost. Making earth observation more affordable has unquestionable benefits as it enables governments to launch their first satellites or public bodies to attempt scientific experiments in an affordable manner, but its opening to the private sector has also created an opportunity for companies to commercialise any observed signal, with or without regard for its sensitivity.

This opportunity was seized by a number of companies to deploy constellations of CubeSats with radio frequency sensors to observe radio frequency signals from space and process the data for surveillance purposes. These satellites are capable of detecting "dark boats" that have disabled their identity broadcast system (AIS) and are actively trying to avoid radar. They are also capable of intercepting any radio frequency that would travel to space such as satellite phone communications or Satellite based internet like Starlink.

Artificial Intelligence (AI) and improved image recognition

The second innovation is the improvement that image quality has seen in the past few years. The same way that CubeSat benefits from fast paced innovation in smartphone chips for computing (chips have become more powerful as smartphones have evolved and CubeSats benefit from this with higher computational power at a lower cost), satellites also benefit from the innovation in sensors and in Artificial Intelligence (AI).

With the explosion of the smartphone market and the heavy reliance of social media on photos, cameras' image quality has indeed drastically improved other the past years, even making it viable to launch smartphones in space. But more interesting is the development of AI. If you're unfamiliar with Artificial Intelligence related to imagery, the gist of it is that using existing data sets of images that have been labelled (e.g.: two boats in the sea), one can train an algorithm to recognize elements in a photo. You can also use similar techniques to improve image quality or detect changes between two images (for example to spot an object moving). Hence, with the rise of technologies such as recognition software, machine learning and other neural networks, it's become much easier to track the movement ot a given objects (such as a boat) or to recognise objects in an image.

Thus, machine learning makes it easier to track a boat in the Mediterranean in an automated way, doing the tedious work of comparing images to try and spot the movement of a boat in a matter of seconds. It also makes the monitoring of a refugee camps or the observation of specific areas much easier, with software being able to constantly process images and alert users when a specific event happens.

The German NGO Space-Eye is developing technology to monitor and analyse satellite imagery to identify migrant boats. It wants to analyse historical footage and investigate whether authorities lived up to their duty. Images are reportedly sourced from private US Earth-imaging company and satellite opertor Planet.

UNHCR’s Project Jetson is another example, computing an index with AI that allows to make short-term predictions of expected migration flows in Somalia based on key variables, such as commodity market prices, rainfall and violent conflict (more examples can be found on the Migration Data Portal).

At their core, these projects use labelled historical images (e.g.: satellite images of a drought in a certain region) to train an algorithm to recognise specific objects or situations. Once the algorithm can detect those with good accuracy, it is fed with live images and can alert the user when the object researched is detected. This allows to process large quantities with way less human resources than a manual analysis would require.

Another innovation worth mentioning is the use of cloud computing for computational task, meaning the satellites don't need to embed super powerful computers and can instead transfer this data to cloud services such as Amazon AWS. This allow better scalability, more computing power and enables satellites to be used for a longer period of time since they won't be limited by the technology they embed.

What this means for the migration sector

The use of satellite and aerial surveillance pose multiple and varied challenges for the migration sector. The developments in the technologies that fall under this type of surveillance raise broader concerns, for example, to automated decision-making, use of facial recognition technology and data sharing.

As has been explored in relation to the use of drones in warfare, surveillance and targeting is on the one hand more individualized and personalised, but on the other hand more and more dehumanised. This relates not only to the physical distance from those who are being monitored but also that software can be similar to that of gaming systems as targets transformed from living people to insignificant icons on computers.

These developments come at a time when the number of children arriving on small boats is reportedly increasing, many unaccompanied and potential victims of trafficking arriving via small boat are said to have been failed by the Home Office.