Category Archives: Amazon AWS

Join Me at SoTec 2014

I will be giving an overview of cloud computing at the 2014 Southland Technology Conference (SoTec 2014) at the Hilton in Long Beach, CA this week.

If you are interested in hearing me speak or learning about other technology topics, please check it out. Here is the link with more information and registration details: http://www.sotecconference.com/.

There will also be a job fair and resume review.

AWS Gotcha: IAmazonS3.CopyObject() Method On S3 Objects Changes ACLs

There are many gotchas with Amazon’s AWS (I suppose that is true with all technologies).  Here is one that plagued me just the other day:

I had a bunch of publicly readable files in an S3 bucket.  I needed to update some headers for all items in the bucket, so I wrote a PowerShell script that calls the AWS .Net SDK CopyObject() method on an instance of the Amazon.S3.AmazonS3Client class. (Of course, I could have just done this in C# but PowerShell seemed easier for my purposes.)

Advanced aside: CopyObject() actually comes from the IAmazonS3 interface, which is implenented in AmazonS3Client. An AmazonS3Client object is returned from the Amazon.AWSClientFactory.CreateAmazonS3Client() factory method as show below.

This is an excerpt from my script, the variables should be self explanatory if you are familiar with AWS:

$s3Config = New-Object Amazon.S3.AmazonS3Config
$s3Config.RegionEndpoint = [Amazon.RegionEndpoint]::USEast1
$s3client = [Amazon.AWSClientFactory]::CreateAmazonS3Client($AccessKey, $SecretKey, $s3Config)

$s3Request = New-Object Amazon.S3.Model.CopyObjectRequest 
$s3Request.MetadataDirective = "REPLACE"
$s3Request.SourceBucket = $s3Bucket
$s3Request.SourceKey = $s3object.Key;
$s3Request.DestinationBucket = $s3Bucket
$s3Request.DestinationKey = $s3object.Key;
$s3Request.Headers.Item('Content-Disposition') =  <new-content-disposition>
$s3Request.Headers.Item('Content-Type') = <new content type>
$s3client.CopyObject($s3Request)

What I learned too late was that CopyObject() removes the public read ACL from my $s3Object.  I ended up needing to clean up my S3 objects to re-enable public read, using another PowerShell script similar to the following:

$s3Config = New-Object Amazon.S3.AmazonS3Config
$s3Config.RegionEndpoint = [Amazon.RegionEndpoint]::USEast1
$s3client = [Amazon.AWSClientFactory]::CreateAmazonS3Client($AccessKey, $SecretKey, $s3Config)

$s3Request = New-Object Amazon.S3.Model.PutACLRequest
            
$s3Request.BucketName = $s3Bucket
$s3Request.Key = $s3object.Key
$s3Request.CannedACL = [Amazon.S3.S3CannedACL]::PublicRead
$s3client.PutACL($s3Request) > $null

I guess the moral of the story is 1) AWS is fraught with best practices that can only be learned by making mistakes, and 2) Test. Test. Test.