How To Upload A Xml File To Aws S3 Java Sdk Example

[AWS S3 CLI] It is easier to manager AWS S3 buckets and objects from CLI. This tutorial explains the basics of how to manage S3 buckets and its objects using aws s3 cli using the following examples:

For quick reference, hither are the commands. For details on how these commands piece of work, read the rest of the tutorial.

# s3 make bucket (create bucket) aws s3 mb s3://tgsbucket --region us-west-two  # s3 remove saucepan aws s3 rb s3://tgsbucket aws s3 rb s3://tgsbucket --force  # s3 ls commands aws s3 ls aws s3 ls s3://tgsbucket aws s3 ls s3://tgsbucket --recursive aws s3 ls s3://tgsbucket --recursive  --man-readable --summarize  # s3 cp commands aws s3 cp getdata.php s3://tgsbucket aws s3 cp /local/dir/data s3://tgsbucket --recursive aws s3 cp s3://tgsbucket/getdata.php /local/dir/data aws s3 cp s3://tgsbucket/ /local/dir/data --recursive aws s3 cp s3://tgsbucket/init.xml s3://backup-bucket aws s3 cp s3://tgsbucket s3://backup-bucket --recursive  # s3 mv commands aws s3 mv source.json s3://tgsbucket aws s3 mv s3://tgsbucket/getdata.php /abode/project aws s3 mv s3://tgsbucket/source.json s3://backup-bucket aws s3 mv /local/dir/data s3://tgsbucket/data --recursive aws s3 mv s3://tgsbucket s3://backup-bucket --recursive  # s3 rm commands aws s3 rm s3://tgsbucket/queries.txt aws s3 rm s3://tgsbucket --recursive  # s3 sync commands aws s3 sync backup s3://tgsbucket aws s3 sync s3://tgsbucket/backup /tmp/backup aws s3 sync s3://tgsbucket s3://backup-saucepan  # s3 bucket website aws s3 website s3://tgsbucket/ --alphabetize-document index.html --error-document error.html  # s3 presign url (default 3600 seconds) aws s3 presign s3://tgsbucket/dnsrecords.txt aws s3 presign s3://tgsbucket/dnsrecords.txt --expires-in 60

one. Create New S3 Bucket

Use mb option for this. mb stands for Make Saucepan.

The following will create a new S3 bucket

$ aws s3 mb s3://tgsbucket make_bucket: tgsbucket

In the above example, the bucket is created in the us-due east-one region, as that is what is specified in the user's config file as shown beneath.

$ true cat ~/.aws/config [profile ramesh] region = us-east-1

To setup your config file properly, use aws configure command as explained hither: fifteen AWS Configure Control Examples to Manage Multiple Profiles for CLI

If the bucket already exists, and you own the bucket, you'll get the following mistake message.

$ aws s3 mb s3://tgsbucket make_bucket failed: s3://tgsbucket An error occurred (BucketAlreadyOwnedByYou) when calling the CreateBucket operation: Your previous request to create the named bucket succeeded and you already ain it.

If the bucket already exists, but owned by some other user, you lot'll get the following error message.

$ aws s3 mb s3://paloalto make_bucket failed: s3://paloalto An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is non available. The bucket namespace is shared past all users of the system. Please select a dissimilar name and try again.

Under some situation, you lot might too get the post-obit error message.

$ aws s3 mb s3://demo-bucket make_bucket failed: s3://demo-bucket An fault occurred (IllegalLocationConstraintException) when calling the CreateBucket performance: The unspecified location constraint is incompatible for the region specific endpoint this request was sent to.

two. Create New S3 Bucket – Unlike Region

To create a saucepan in a specific region (different than the one from your config file), and so utilize the –region option as shown below.

$ aws s3 mb s3://tgsbucket --region united states-w-2 make_bucket: tgsbucket

iii. Delete S3 Bucket (That is empty)

Use rb option for this. rb stands for remove bucket.

The following deletes the given bucket.

$ aws s3 rb s3://tgsbucket remove_bucket: tgsbucket

If the bucket you are trying to delete doesn't exists, you'll get the post-obit fault message.

$ aws s3 rb s3://tgsbucket1 remove_bucket failed: s3://tgsbucket1 An error occurred (NoSuchBucket) when calling the DeleteBucket operation: The specified saucepan does not exist

iv. Delete S3 Bucket (And all its objects)

If the bucket contains some object, you lot'll become the following fault message:

$ aws s3 rb s3://tgsbucket remove_bucket failed: s3://tgsbucket An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty

To delete a bucket along with all its objects, use the –force option as shown below.

$ aws s3 rb s3://tgsbucket --force delete: s3://tgsbucket/demo/getdata.php delete: s3://tgsbucket/ipallow.txt delete: s3://tgsbucket/demo/servers.txt delete: s3://tgsbucket/demo/ remove_bucket: tgsbucket

5. List All S3 Buckets

To view all the buckets endemic by the user, execute the following ls command.

$ aws s3 ls 2019-02-06 xi:38:55 tgsbucket 2018-12-18 18:02:27 etclinux 2018-12-08 eighteen:05:15 readynas .. ..

In the above output, the timestamp is the date the saucepan was created. The timezone was adjusted to be displayed to your laptop'southward timezone.

The following command is same as the above:

aws s3 ls s3://

6. List All Objects in a Bucket

The following command displays all objects and prefixes nether the tgsbucket.

$ aws s3 ls s3://tgsbucket                            PRE config/                            PRE data/ 2019-04-07 11:38:twenty         xiii getdata.php 2019-04-07 11:38:20       2546 ipallow.php 2019-04-07 11:38:xx          9 license.php 2019-04-07 11:38:xx       3677 servers.txt

In the above output:

Inside the tgsbucket, there are two folders config and data (indicated past PRE)
PRE stands for Prefix of an S3 object.
Inside the tgsbucket, we take 4 files at the / level
The timestamp is when the file was created
The second column display the size of the S3 object

Notation: The above output doesn't display the content of sub-folders config and data

7. Listing all Objects in a Saucepan Recursively

To display all the objects recursively including the content of the sub-folders, execute the post-obit command.

$ aws s3 ls s3://tgsbucket --recursive 2019-04-07 11:38:19       2777 config/init.xml 2019-04-07 xi:38:20         52 config/support.txt 2019-04-07 11:38:20       1758 information/database.txt 2019-04-07 11:38:20         13 getdata.php 2019-04-07 11:38:xx       2546 ipallow.php 2019-04-07 11:38:twenty          ix license.php 2019-04-07 xi:38:xx       3677 servers.txt

Note: When you are listing all the files, notice how there is no PRE indicator in the 2nd cavalcade for the folders.

viii. Total Size of All Objects in a S3 Bucket

You can identify the total size of all the files in your S3 saucepan by using the combination of post-obit three options: recursive, human-readable, summarize

Note: The post-obit displays both total file size in the S3 bucket, and the total number of files in the s3 bucket

$ aws s3 ls s3://tgsbucket --recursive  --homo-readable --summarize 2019-04-07 xi:38:19    2.7 KiB config/init.xml 2019-04-07 11:38:20   52 Bytes config/back up.txt 2019-04-07 xi:38:20    1.7 KiB data/database.txt 2019-04-07 11:38:20   thirteen Bytes getdata.php 2019-04-07 xi:38:xx    2.five KiB ipallow.php 2019-04-07 xi:38:20    nine Bytes license.php 2019-04-07 11:38:20    three.6 KiB servers.txt  Total Objects: 7    Total Size: ten.6 KiB

In the above output:

recursive option brand sure that it displays all the files in the s3 bucket including sub-folders
homo-readable displays the size of the file in readable format. Possible values you'll meet in the 2nd column for the size are: Bytes/MiB/KiB/GiB/TiB/PiB/EiB
summarize options make sure to display the last two lines in the to a higher place output. This indicates the total number of objects in the S3 bucket and the full size of all those objects

9. Request Payer List

If a specific bucket is configured as requester pays buckets, then if you are accessing objects in that bucket, you sympathise that you lot are responsible for the payment of that request access. In this case, bucket owner doesn't have to pay for the admission.

To indicate this in your ls command, you lot'll have to specify –request-payer option as shown below.

$ aws s3 ls s3://tgsbucket --recursive --request-payer requester 2019-04-07 11:38:19       2777 config/init.xml 2019-04-07 11:38:20         52 config/support.txt 2019-04-07 11:38:twenty       1758 information/database.txt 2019-04-07 11:38:20         13 getdata.php 2019-04-07 11:38:20       2546 ipallow.php 2019-04-07 eleven:38:20          ix license.php 2019-04-07 11:38:xx       3677 servers.txt

For signed URL, make certain to include x-amz-asking-payer=requester in the asking

10. Re-create Local File to S3 Bucket

In the following case, we are copying getdata.php file from local laptop to S3 bucket.

$ aws s3 cp getdata.php s3://tgsbucket upload: ./getdata.php to s3://tgsbucket/getdata.php

If you desire to copy the getdata.php to a S3 bucket with a different name, practise the following

$ aws s3 cp getdata.php s3://tgsbucket/getdata-new.php upload: ./getdata.php to s3://tgsbucket/getdata-new.php

For the local file, you tin also specify the total path as shown beneath.

$ aws s3 cp /home/project/getdata.php s3://tgsbucket upload: ../../home/project/getdata.php to s3://tgsbucket/getdata.php

11. Copy Local Folder with all Files to S3 Saucepan

In this case, we are copying all the files from the "data" folder that is under /dwelling house/projects directory to S3 bucket

$ cd /home/projects  $ aws s3 cp data s3://tgsbucket --recursive upload: data/parameters.txt to s3://tgsbucket/parameters.txt upload: information/common.txt to s3://tgsbucket/common.txt ..

In the in a higher place example, note that only the files from the local data/ folder is getting uploaded. Not the folder "data" itself

If y'all like to upload the data folder from local to s3 bucket equally data folder, and then specify the folder proper name after the bucket name as shown beneath.

$ aws s3 cp data s3://tgsbucket/information --recursive upload: data/parameters.txt to s3://tgsbucket/data/parameters.txt upload: data/common.txt to s3://tgsbucket/information/common.txt .. ..

12. Download a File from S3 Bucket

To download a specific file from an S3 bucket do the following. The following copies getdata.php from the given s3 bucket to the current directory.

$ aws s3 cp s3://tgsbucket/getdata.php . download: s3://tgsbucket/getdata.php to ./getdata.php

You tin can download the file to the local machine with in a unlike name as shown below.

$ aws s3 cp s3://tgsbucket/getdata.php getdata-local.php download: s3://tgsbucket/getdata.php to ./getdata-local.php

Download the file from S3 bucket to a specific folder in local machine as shown beneath. The post-obit volition download getdata.php file to /home/project folder on local machine.

$ aws s3 cp s3://tgsbucket/getdata.php /abode/project/ download: s3://tgsbucket/getdata.php to ../../home/project/getdata.php

13. Download All Files Recursively from a S3 Bucket (Using Copy)

The following will download all the files from the given bucket to the electric current directory on your laptop.

$ aws s3 cp s3://tgsbucket/ . --recursive download: s3://tgsbucket/getdata.php to ./getdata.php download: s3://tgsbucket/config/init.xml ./config/init.xml ..

If you want to download all the files from a S3 bucket to a specific folder locally, please specify the full path of the local directory as shown below.

$ aws s3 cp s3://tgsbucket/ /home/projects/tgsbucket --recursive download: s3://tgsbucket/getdata.php to ../../dwelling/projects/tgsbucket/getdata.php download: s3://tgsbucket/config/init.xml to ../../home/projects/tgsbucket/config/init.xml ..

In the to a higher place command, if the tgsbucket folder doesn't exists nether /abode/projects, it will create it automatically.

fourteen. Re-create a File from One Saucepan to Another Saucepan

The following command will copy the config/init.xml from tgsbucket to backup bucket equally shown below.

$ aws s3 cp s3://tgsbucket/config/init.xml s3://fill-in-bucket copy: s3://tgsbucket/config/init.xml to s3://backup-saucepan/init.xml

In the in a higher place case, eventhough init.xml file was nether config folder in the source saucepan, on the destination bucket, it copied the init.xml file to the elevation-level / in the backup-saucepan.

If you want to copy the same folder from source and destination along with the file, specify the binder name in the desintation bucketas shown below.

$ aws s3 cp s3://tgsbucket/config/init.xml s3://backup-bucket/config copy: s3://tgsbucket/config/init.xml to s3://backup-saucepan/config/init.xml

If the destination bucket doesn't exist, you'll get the following mistake bulletin.

$ aws s3 cp s3://tgsbucket/test.txt s3://fill-in-bucket-777 re-create failed: s3://tgsbucket/test.txt to s3://backup-saucepan-777/test.txt An error occurred (NoSuchBucket) when calling the CopyObject operation: The specified bucket does non exist

15. Copy All Files Recursively from One Saucepan to Another

The following will re-create all the files from the source bucket including files under sub-folders to the destination bucket.

$ aws s3 cp s3://tgsbucket s3://backup-bucket --recursive copy: s3://tgsbucket/getdata.php to s3://backup-bucket/getdata.php copy: s3://tgsbucket/config/init.xml s3://backup-bucket/config/init.xml ..

16. Motion a File from Local to S3 Bucket

When yous motion file from Local machine to S3 bucket, as you would expect, the file will exist physically moved from local auto to the S3 bucket.

$ ls -l source.json -rw-r--r--  i ramesh  sysadmin  1404 Apr  ii thirteen:25 source.json  $ aws s3 mv source.json s3://tgsbucket move: ./source.json to s3://tgsbucket/source.json

As yous see the file doesn't exists on the local machine later the motility. Its only on S3 bucket at present.

$ ls -l source.json ls: source.json: No such file or directory

17. Move a File from S3 Bucket to Local

The following is reverse of the previou example. Hither, the file will exist moved from S3 bucket to local automobile.

As you see below, the file at present exists on the s3 bucket.

$ aws s3 ls s3://tgsbucket/getdata.php 2019-04-06 06:24:29       1758 getdata.php

Motility the file from S3 saucepan to /dwelling/project directory on local machine.

$ aws s3 mv s3://tgsbucket/getdata.php /dwelling house/projection move: s3://tgsbucket/getdata.php to ../../../home/project/getdata.php

Afterwards the move, the file doesn't exists on S3 bucketanymore.

$ aws s3 ls s3://tgsbucket/getdata.php

18. Move a File from One S3 Bucket to Some other S3 Bucket

Before the move, the file source.json is in tgsbucket.

$ aws s3 ls s3://tgsbucket/source.json 2019-04-06 06:51:39       1404 source.json

This file is not in backup-bucket.

$ aws s3 ls s3://backup-saucepan/source.json $

Motility the file from tgsbucketto backup-bucket.

$ aws s3 mv s3://tgsbucket/source.json s3://backup-bucket move: s3://tgsbucket/source.json to s3://fill-in-bucket/source.json

Now, the file is merely on the backup-bucket.

$ aws s3 ls s3://tgsbucket/source.json $  $ aws s3 ls s3://fill-in-bucket/source.json 2019-04-06 06:56:00       1404 source.json

19. Move All Files from a Local Folder to S3 Bucket

In this example, the following files are under data folder.

$ ls -one data dnsrecords.txt parameters.txt dev-setup.txt error.txt

The following moves all the files in the data directory on local machine to tgsbucket

$ aws s3 mv information s3://tgsbucket/information --recursive move: data/dnsrecords.txt to s3://tgsbucket/data/dnsrecords.txt move: information/parameters.txt to s3://tgsbucket/information/parameters.txt move: data/dev-setup.txt to s3://tgsbucket/data/dev-setup.txt motility: data/error.txt to s3://tgsbucket/information/error.txt

20. Move All Files from S3 Saucepan to Local Folder

In this example, the localdata binder is currently empty.

$ ls -1 localdata $

The following will move all the files in the S3 bucketunder data binder to localdata folder on your local automobile.

$ aws s3 mv s3://tgsbucket/information/ localdata --recursive move: s3://tgsbucket/data/dnsrecords.txt to localdata/dnsrecords.txt move: s3://tgsbucket/data/parameters.txt to localdata/parameters.txt movement: s3://tgsbucket/data/dev-setup.txt to localdata/dev-setup.txt move: s3://tgsbucket/data/fault.txt to localdata/error.txt

Here is the output after the above move.

$ aws s3 ls s3://tgsbucket/data/ $  $ ls -1 localdata dnsrecords.txt parameters.txt dev-setup.txt error.txt

21. Movement All Files from One S3 Bucket to Another S3 Bucket

Utilize the recursive selection to move all files from one bucket to some other as shown below.

$ aws s3 mv s3://tgsbucket s3://fill-in-bucket --recursive motility: s3://tgsbucket/dev-setup.txt to s3://backup-saucepan/dev-setup.txt move: s3://tgsbucket/dnsrecords.txt to s3://backup-saucepan/dnsrecords.txt move: s3://tgsbucket/error.txt to s3://fill-in-bucket/mistake.txt move: s3://tgsbucket/parameters.txt to s3://backup-bucket/parameters.txt

22. Delete a File from S3 Saucepan

To delete a specific file from a S3 bucket, use the rm selection as shown below. The post-obit volition delete the queries.txt file from the given S3 bucket.

$ aws s3 rm s3://tgsbucket/queries.txt delete: s3://tgsbucket/queries.txt

23. Delete All Objects from S3 buckets

When yous specify rm option but with a bucket name, it doesn't do anything. This will not delete any file from the bucket.

aws s3 rm s3://tgsbucket

To delete all the files from a S3 bucket, use the –recursive option as bear witness nbelow.

$ aws s3 rm s3://tgsbucket --recursive delete: s3://tgsbucket/dnsrecords.txt delete: s3://tgsbucket/common.txt delete: s3://tgsbucket/parameters.txt delete: s3://tgsbucket/config/init.xml ..

24. Sync files from Laptop to S3 Bucket

When yous apply sync command, it volition recursively copies only the new or updated files from the source directory to the destination.

The post-obit will sync the files from backup directory in local car to the tgsbucket.

$ aws s3 sync backup s3://tgsbucket upload: backup/docker.sh to s3://tgsbucket/docker.sh upload: fill-in/address.txt to s3://tgsbucket/address.txt upload: backup/display.py to s3://tgsbucket/display.py upload: fill-in/getdata.php to s3://tgsbucket/getdata.php

If you desire to sync information technology to a subfolder called backup on the S3 bucket, and so include the binder name in the s3 bucket as shown beneath.

$ aws s3 sync backup s3://tgsbucket/fill-in upload: fill-in/docker.sh to s3://tgsbucket/fill-in/docker.sh upload: fill-in/accost.txt to s3://tgsbucket/backup/address.txt upload: backup/brandish.py to s3://tgsbucket/backup/display.py upload: backup/getdata.php to s3://tgsbucket/backup/getdata.php

Once y'all do the sync once, if you run the control immediately again, it will not do annihilation, equally in that location is no new or updated files on the local backup directory.

$ aws s3 sync backup s3://tgsbucket/backup $

Let united states of america create a new file on the local machine for testing.

echo "New file" > backup/newfile.txt

Now when yous execute the sync, it volition sync only this new file to the S3 bucket.

$ aws s3 sync backup s3://tgsbucket/backup upload: fill-in/newfile.txt to s3://tgsbucket/backup/newfile.txt

25. Sync File from S3 bucket to Local

This is reverse of the previous example. Here, we are syncing the files from the S3 bucket to the local machine.

$ aws s3 sync s3://tgsbucket/backup /tmp/fill-in download: s3://tgsbucket/backup/docker.sh to ../../tmp/backup/docker.sh download: s3://tgsbucket/backup/display.py to ../../tmp/backup/display.py download: s3://tgsbucket/backup/newfile.txt to ../../tmp/backup/newfile.txt download: s3://tgsbucket/fill-in/getdata.php to ../../tmp/backup/getdata.php download: s3://tgsbucket/backup/address.txt to ../../tmp/backup/address.txt

26. Sync Files from one S3 Bucket to Another S3 Saucepan

The following example syncs the files from one tgsbucket to backup-bucket

$ aws s3 sync s3://tgsbucket s3://backup-bucket re-create: s3://tgsbucket/backup/newfile.txt to s3://backup-bucket/fill-in/newfile.txt copy: s3://tgsbucket/backup/display.py to s3://fill-in-bucket/fill-in/display.py copy: s3://tgsbucket/backup/docker.sh to s3://backup-bucket/backup/docker.sh re-create: s3://tgsbucket/backup/address.txt to s3://backup-bucket/backup/address.txt re-create: s3://tgsbucket/backup/getdata.php to s3://backup-bucket/backup/getdata.php

27. Set S3 bucket as a website

You lot can also make S3 bucket to host a static website as shown below. For this, you need to specify both the alphabetize and fault document.

aws s3 website s3://tgsbucket/ --alphabetize-document index.html --fault-document fault.html

This bucket is in us-eastward-1 region. So, in one case you've done the above, you can access the tgsbucket as a website using the post-obit URL: http://tgsbucket.s3-website-us-east-ane.amazonaws.com/

For this to piece of work properly, make sure public admission is attack this S3 bucket, equally this acts equally a website now.

28. Presign URL of S3 Object for Temporary Admission

When you presign a URL for an S3 file, anyone who was given this URL tin can retrieve the S3 file with a HTTP Become request.

For example, if you want to give access to the dnsrecords.txt file to someone temporarily, presign this specific S3 object as shown below.

$ aws s3 presign s3://tgsbucket/dnsrecords.txt https://tgsbucket.s3.amazonaws.com/error.txt?AWSAccessKeyId=AAAAAAAAAAAAAAAAAAAA&Expires=1111111111&Signature=ooooooooooo%2Babcdefghijlimmm%3A

The output of the higher up command volition exist a HTTPS url, which you lot can paw it out someone who should exist able to download the dnsrecords.txt file from your S3 bucket.

The above URL volition be valid by default for 3600 seconds (1 60 minutes).

If yous want to specify a curt expirty time, use the following expires-in pick. The following volition create a presigned URL that is valid only for 1 minute.
–expires-in (integer) Number of seconds until the pre-signed URL expires. Default is 3600 seconds.

$ aws s3 presign s3://tgsbucket/dnsrecords.txt --expires-in 60 https://tgsbucket.s3.amazonaws.com/error.txt?AWSAccessKeyId=AAAAAAAAAAAAAAAAAAAA&Expires=1111111111&Signature=ooooooooooo%2Babcdefghijlimmm%3A

If someone tries to access the URL afterwards the death time, they'll see the following AccessDenied message.

<Error> <Code>AccessDenied</Code> <Message>Request has expired</Message> <Expires>2019-04-07T11:38:12Z</Expires> <ServerTime>2019-04-07T11:38:21Z</ServerTime> <RequestId>1111111111111111</RequestId> <HostId> mmmmmmmmmm/ggggggggg </HostId> </Fault>

Source: https://www.thegeekstuff.com/2019/04/aws-s3-cli-examples/

Posted by: carrollcieved.blogspot.com