Side Project Chronicles, ep. 4: CLI/SDK

tl;dr

This post got a little long, so here are the learnings:

  1. Any of the AWS SDKs, including boto3, as well as commercial IaC, all have Lightsail libraries.
  2. The user id and user secret created in Lightsail do not have console access, so to use an SDK you need to create a user via the regular IAM.
  3. Lightsail libraries in SDKs are limited to only the functionality of the Lightsail console.
  4. Lightsail S3 buckets do not appear in the ListBuckets results from regular S3 components in the SDKs, but can be addressed directly using a regular S3 library if you know the bucket name.
  5. When using a regular S3 library, the functionality is again limited to what Lightsail supports; attempting an unsupported action returns “Access Denied” exception.
  6. The async .NET SDKs for .NET 5+ do not implement all the methods found in the full .NET Framework versions.  I switched to boto3 and Python rather than install .NET 4.x to test ListBuckets and similar actions; see #4 for how that worked out.

The Full Adventure

The Lightsail console provides us a lot of functionality, but it’s not easy to audit the changes we make using the console.  The console is a manual process and we have to remember to always check the same settings; hence why IaC is a best practice.  Based on our look at the S3 bucket, we know more is happening via Lightsail than we can see, and we assume good decisions are being made.  Something specific I’d like to check is if objects are encrypted at rest.  Since a lot of automated compliance tooling uses the API or and SDK to check adherence to enterprise rules, we want to make sure we can use these to access the settings we’re interested in.  As it turns out we have a number of options for SDK/CLI/IaC for Lightsail:

(SDKs are also available for several languages other than .NET)

I want to try the AWS SDK for .NET, since that’s my most native programming language.  The AWS SDK specifically for Lightsail is available via Nuget, which describes Lightsail as “[a]n extremely simplified VM creation and management service.”  Despite that outdated description, the SDK is current.

The Lightsail S3 buckets were not visible in the usual S3 console, so I wanted to see if they are visible to the CLI or SDK.  AWS has an example of how to list buckets with the SDK, at https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/dotnetv3/S3/ListBucketsExample/ListBuckets.cs

As it turns out, the bucket access keys we created in the Lightsail console do not grant permissions to use CLI/SDK.  This is one instance where we need to use the normal AWS control panel rather than the Lightsail control panel, and create an IAM user with more privileged  permissions (https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-bucket-management-policies).

With a more privileged set of user credentials in place, we can run the AWS sample, and see that the Lightsail buckets are not listed in the response.  That makes sense, since we didn’t see them in the API, but it’s good to check.

If we know the name of the bucket, we can access it directly, but the actions we can perform are limited.  It was at this point I realized not all functions in the .NET Framework version have been implemented in the .NET version; instead of installing .NET 4.x, I switched to python boto3.  What I found is, when using a regular S3 library, the you can list_objects but not get_bucket_encryption.  Get_bucket_encryption returns an Access Denied error, even when using secrets for the root user.

To wrap this all up, you can use either a Lightsail SDK, or a regular S3 SDK, to work with Lightsail buckets.  Either way, functionality is limited to what Lightsail supports.  You’ll just have to take it on faith that AWS’ defaults are secure enough for your needs.  It’s unlikely policy scanning tools can detect or validate best practices on your Lightsail buckets.