How to download EODATA file using boto3 on ESA HPC

Note

LEGACY ARTICLE

This article is marked as a legacy document and may not reflect the latest information. Please refer to the following article: How to access EODATA using boto3 on ESA HPC

Download particular Landsat-5 image.

Warning

We strongly recommend using virtualenv for isolating python packages. Configuration tutorial is available here:

How to install Python virtualenv or virtualenvwrapper on ESA HPC.

If virtualenv is activated:

(myvenv) eouser@vm01:~$ pip3 install boto3

Or if we install package globally:

eouser@vm01:~$ sudo pip3 install boto3

To get the credentials for your VM, see article How to get credentials used for accessing EODATA on a cloud VM on ESA HPC.

As an example, let us suppose that the credentials you got from that article are

BSIYKQYCZKUIMXXUBPDAADDAAAADDV:kksHunhhnTreMIxyzePGqMavSBuwDbOxHRAYhahkQex

Script for downloading one .png file

Remember that you cannot change the output file extension. If you do so, the file content would be empty.

import boto3

access_key='<YOUR_ACCESS_KEY_FROM_/etc/passwd-s3fs>'
secret_key='<YOUR_SECRET_KEY_FROM_/etc/passwd-s3fs>'
key='Landsat-5/TM/L1T/2011/11/16/LS05_RMPS_TM__GTC_1P_20111116T100042_20111116T100111_147386_0194_0035_4BF1/LS05_RMPS_TM__GTC_1P_20111116T100042_20111116T100111_147386_0194_0035_4BF1.BP.PNG'
host='https://eodata.cloudferro.com'

s3=boto3.resource('s3',aws_access_key_id=access_key,
aws_secret_access_key=secret_key, endpoint_url=host,)

bucket=s3.Bucket('DIAS')

bucket.download_file(key, '/home/eouser/image.png')

When substituted the values of credentials, it would look like this:

        import boto3

        access_key='<BSIYKQYCZKUIMXXUBPDAADDAAAADDV>'
        secret_key='kksHunhhnTreMIxyzePGqMavSBuwDbOxHRAYhahkQex'
# the rest of the code from above
...

Save your file with .py extension and run with the ‘python3 <filename.py>’ command in your terminal. For example:

(boto3) eouser@vm01:~$ python3 eodownload.py

And here’s result:

(boto3) eouser@vm01:~$ python3 eodownload.py

(boto3) eouser@vm01:~$ ls

eodownload.py  image.png