Vincent GODARD - V1- 31/01/2024
Freely inspired by :
Abdishakur Hassan, 2019, Satellite imagery access and analysis in Python & Jupyter notebooks: https://towardsdatascience.com/satellite-imagery-access-and-analysis-in-python-jupyter-notebooks-387971ece84b
Gabriela Reis, 2024, Querying and Downloading Sentinel Satellite Data with Python: https://medium.com/@gabrielagodinho/querying-and-downloading-sentinel-satellite-data-with-python-80573e4b16f5
Mahyar Aboutalebi, 2023, Mahyar AboutalebiDownloading Sentinel-2 Imagery in Python with Google Colab (Updated Nov 2023):https://medium.com/@mahyar.aboutalebi/downloading-sentinel-2-imagery-in-python-with-google-colab-updated-nov-2023-f21d75a92407
Sources :
Data ESA (https://www.esa.int/ESA_Multimedia/Images/2017/03/Sentinel-2_global_coverage);
The Level-2A product (https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/overview);
Sentinel-2 Products Specification Document (https://sentinel.esa.int/documents/247904/685211/sentinel-2-products-specification-document);
The Copernicus Browser (https://dataspace.copernicus.eu/browser).
Compressed file to download => here. (not for the moment).
Select the coordinates of the 4 angles of your AOI.
Pay attention to the order of longitude and latitude.
If your coordinates are in sexagesimal degrees, convert them to decimal degrees (for example here https://www.fcc.gov/media/radio/dms-decimal ).
Your AOI probably has something like this (long _ lat):
NW : 38.68 9.13
NE : 38.79 9.13
SE : 38.79 9.01
SW : 38.68 9.01
And also the approximate center of AOI polygon:
38.734 9.074
Before you can download Sentinelsat data, you must log on to the Copernicus Open Access Hub (https://dataspace.copernicus.eu/) with an username and a password (to set up in Sign up). Things to do before continuing!
Be careful, your future username will be your "email address" and your password, your "password".
To view them, visit the following website again: https://dataspace.copernicus.eu/.
Then "Explore data/Copernicus Browser"
Next, click “Login” and select “Register”.
When you are in the browser:
Center yourself on your AOI.
Select:
Date: Time Range
From 2023/10/01
Until 2024/01/29
Max. cloud coverage: 20%
Data collections:
Sentinel-2 L2A
And click on "Find products within selected time range"
○ How many image(s) were found?
If necessary, change the “Weather Range” or “Maximum Cloud Cover”.
## Loading Libraries
!pip install sentinelsat
!pip install creds # Creds is a library that simplifies the management of user accounts and their credentials on Linux, FreeBSD and OpenBSD.
!pip install rasterio
!pip install folium
!apt install gdal-bin python-gdal python3-gdal
!apt install python3-rtree
#!pip install git+git://github.com/geopandas/geopandas.git
!pip install --user geopandas
!pip install descartes
## Importing libraries
import folium
import os
import numpy as np
import requests
import json
import creds
from sentinelsat import SentinelAPI, read_geojson, geojson_to_wkt
import geopandas as gpd
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from shapely.geometry import MultiPolygon, Polygon
import rasterio as rio
from rasterio.plot import show
import rasterio.mask
import fiona
After installing and importing the libraries, we filter the Sentinel data catalog with different parameters: satellite name, product level (S2MSI2A for surface reflectance), AOI (point or polygon), and start and end dates.
To help you filter with the API: https://sentinelsat.readthedocs.io/en/latest/api_overview.html
## Catalog address (since October 2023)
url_dataspace = "https://catalogue.dataspace.copernicus.eu/odata/v1"
## Filter settings (gentle reminder => by dictionary where {key:value})
satellite = "SENTINEL-2"
level = "S2MSI2A"
cloud_cover_max = 0.2
aoi_point ="POINT(38.734 9.074)" # approximate center of AOI polygon
# aoi_polygon = "POLYGON ((38.68 9.13, 38.79 9.13, 38.79 9.01, 38.68 9.01, 38.68 9.13))" # not useful at this stage, see TD21.
start_date = "2023-10-01"
end_date = "2024-01-18"
start_date_full = start_date+"T00:00:00.000Z"
end_date_full = end_date +"T00:00:00.000Z"
We can now query to retrieve the list of available images. You must filter the Dataframe to only include products with the “online” status:
## The request (with {dictionary})
query = f"{url_dataspace}/Products?$filter=Collection/Name eq '{satellite}' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{level}') and OData.CSC.Intersects(area=geography'SRID=4326;{aoi_point}') and ContentDate/Start gt {start_date_full} and ContentDate/Start lt {end_date_full}"
response = requests.get(query).json()
result = pd.DataFrame.from_dict(response["value"])
## Filter records where 'online' column is True
result = result[result['Online'] == True]
## print first 30 results
result.head(30)
There are more images than on the website!
We'll look at this in a spreadsheet.
## Export the query result to a CSV file:
result.to_csv('data/result_S2MSI2A.csv', index=False)
Open the CSV file with Libre Office.
Why isn't it similar? (Online?)
Which are the most complete?
Choose a high-contrast image. We'll now download a complete scene.
This method is from the Support of Dataspace Copernicus.
You need a product ID which can be found in the response to your previous product search (see CSV file).
To obtain your token, you must provide your Copernicus Data Space Ecosystem username (email) and password. In this example, we import them from a Python file called "creds.py" and import the credentials as variables in the get_access_token() function (to run the next cell you will need to create this file and save in the same folder, like this Notebook).
## Import credentials
from creds import *
It's more secure than write your username and your pwd in the script!
## Import the credentials as variables
def get_access_token(username: str, password: str) -> str:
data = {
"client_id": "cdse-public",
"username": username,
"password": password,
"grant_type": "password",
}
try:
r = requests.post("https://identity.dataspace.copernicus.eu/auth/realms/CDSE/protocol/openid-connect/token",
data=data,
)
r.raise_for_status()
except Exception as e:
raise Exception(
f"Access token creation failed. Reponse from the server was: {r.json()}"
)
return r.json()["access_token"]
access_token = get_access_token(username, password)
Rerun the cell with Import if the "unknown username" error appears.
## If you want to know how is your credential (this step is optional, added FYI by support)
print (access_token)
Using the product ID and access token, you can download the product now.
## Use the Id value read in the CSV file (with the best contrast) => 3ba953a8-1407-4aff-9910-e1f7d6a5d924
url = f"https://zipper.dataspace.copernicus.eu/odata/v1/Products(3ba953a8-1407-4aff-9910-e1f7d6a5d924)/$value"
headers = {"Authorization": f"Bearer {access_token}"}
session = requests.Session()
session.headers.update(headers)
response = session.get(url, headers=headers, stream=True)
with open("data/productS2B_MSIL2A_20231005.zip", "wb") as file:
for chunk in response.iter_content(chunk_size=8192):
if chunk:
file.write(chunk)
There is several information in the zip file:
First, open the “.jpg” file. What is this?
Next, open "productS2B_MSIL2A_20231005.zip\S2A_MSIL2A_20231005T073741_N0509_R092_T37PDL_20231005T115451.SAFE\HTML\UserProduct_index.html. What is this?
What do you find in:
○ productS2B_MSIL2A_20231005.zip\S2A_MSIL2A_20231005T073741_N0509_R092_T37PDL_20231005T115451.SAFE\GRANULE\L2A_T37PDL_A043275_20231005T074627\IMG_DATA\R10m?
○ productS2B_MSIL2A_20231005.zip\S2A_MSIL2A_20231005T073741_N0509_R092_T37PDL_20231005T115451.SAFE\GRANULE\L2A_T37PDL_A043275_20231005T074627\IMG_DATA\R20m
○ productS2B_MSIL2A_20231005.zip\S2A_MSIL2A_20231005T073741_N0509_R092_T37PDL_20231005T115451.SAFE\GRANULE\L2A_T37PDL_A043275_20231005T074627\IMG_DATA\R60m