Copying multiple directories in a single Gradle task with up-to-date checking
December 11th, 2014
How to write a single Gradle task to copy multiple directories and still use up-to-date checking.
We’re always looking to get our software to run faster. For web sites, part of that equation is to move static content to a content delivery network (CDN). A proper CDN is designed to serve static content fast, have multiple geographic domains and long cache times. The load on the application server is lessened, giving more bandwidth for application use rather than static content. This post explores using the Grails cdn-asset plugin to copy and use static resources on the Amazon S3 service. Once configured it makes hosting static content on S3 easy.
Note that the Grails plugin discussed here only supports Amazon S3 at the time of this writing, cdn-asset-pipeline version 0.4.1. If you don’t want to use S3 then this post won’t help you, unless you want to contribute to the plugin and add your chosen provider.
The asset-pipeline plugin manages static resources in your Grails application. It does some very nice things like collecting all similar types into a common file (application.js, application.css, etc.), minification, filenames based on content hash, etc. In development mode these files are broken out to enable reloading without restarting the server. In production the assets are combined and optimized.
The asset-pipeline plugin replaces the capability in the resources plugin. The asset-pipeline plugin supports Grails 2.0 and greater and is the default in Grails 2.4.
What we want to do is move the production resources to a CDN (in this case Amazon S3) and have the application point to the CDN for the static resources. This will improve performance for the client and reduce load on the server. The cdn-asset-pipeline plugin will do that for us.
You will need an Amazon account to access the AWS services. If you’re new to AWS you’ll get some free space and bandwidth for the first year. Go to the S3 Console. Create a new bucket for your static resources. It’s best to have a separate bucket for security reasons. If you make a mistake you can remove all the files and upload them again, nothing is lost. If your bucket is comprised, you can replace the static resources.
Configuring the permissions gave me the biggest trouble. You will want a separate user account for uploading to the bucket. Definitely don’t use your Amazon “root” account and don’t share accounts. If you’re going to upload from different systems, for example your development environment and a CI/CD environment, create separate users for those as well. You can then revoke access from a particular system if needed. In the AWS Console, click IAM. If you haven’t used IAM it will step you through a few things to secure your account. After configuring IAM, follow these steps to create new users and give them upload access to your bucket.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::mybucket/*”, "arn:aws:s3:::mybucket"
]
}
]
}
Add the plugin to your Grails application:
plugins {
compile ':cdn-asset-pipeline:0.4.1'
}
An example configuration is provided below. Note that this is only an example, it is possible to publish to multiple CDNs. See the cdn-asset-pipeline documentation for details.
def appName = grails.util.Metadata.current.'app.name'
def appVersion = grails.util.Metadata.current.'app.version'
def cdnAccessKey = System.getenv('CDN_AWS_ACCESS_KEY_ID')
def cdnSecretKey = System.getenv('CDN_AWS_SECRET_ACCESS_KEY')
if (cdnAccessKey && cdnSecretKey) {
grails {
assets {
cdn {
provider = 's3' // Karman provider
directory = ‘mybucket’
accessKey = cdnAccessKey
secretKey = cdnSecretKey
storagePath = "assets/${appName}-${appVersion}/"
region = 'us-east-1'
expires = 365 // Expires in 1 year (value in days)
gzip = true // save compressed files, improves download speed
}
}
}
}
environments { production {
grails.assets.url = "https://s3.amazonaws.com/mybucket/assets/${appName}-${appVersion}/"
}}
The storage path and therefore the S3 URL can be configured in Config.groovy with Groovy. There are two configurations that are affected by the storage path: the upload configuration and the assets URL. See “grails.assets.cdn.storagePath” and “grails.assets.url” above. The recommended approach is to include the application version.
Amazon S3 allows each resource to provide cache control headers. This is configured with the “expires” key shown above. A long expiry time can be set because the assets-pipeline plugin creates files with a hash based on the content.
Credentials are configured using the “accessKey” and “secretKey” properties. For security purposes, it is recommended to maintain credentials in environment variables. Never commit credentials to a public repository. In this case, the credential user name is kept in CDNAWSACCESSKEYID and the secret in CDNAWSSECRETACCESSKEY.
All assets produced by the asset-pipeline plugin are included in the CDN upload. The number of assets can be trimmed by excluding partial assets, i.e. assets that aren’t directly accessed by the application. An example follows. It assumes assets such as angularjs and bootstrap are included in other files, such as application.js and application.css. Make sure to tailor this to your configuration or your application won’t find the files.
grails.assets.excludes = ["**/*.less","angularjs/**","animate.css","bootstrap*","jquery/*"]
grails.assets.plugin."twitter-bootstrap".excludes = ["**/*.less"]
grails.assets.plugin."twitter-bootstrap".includes = ["bootstrap.less"]
grails.assets.plugin."font-awesome-resources".excludes = ["font-awesome/**/*.less"]
grails.assets.plugin."font-awesome-resources".includes = ["font-awesome/font-awesome.less"]
$ grails asset-cdn-push
That’s it! The plugin will build assets and upload them. It is smart enough to only upload files not already present in the CDN. Make sure to add this command to your CI/CD environment. Try it by running your application in production mode:
$ grails prod run-war
Using a CDN is a best practice to improve the user experience of your application. For Grails applications, the combination of the asset-pipeline and cdn-asset-pipeline plugins make it easy to maintain your static assets on one or more CDNs.
How to write a single Gradle task to copy multiple directories and still use up-to-date checking.
A workaround for a couple of Grails 2.x bugs and changes that have made it difficult to sort criteria case-insensitively when using child/nested properties.
Fat jar deployments in Vert.x can simplify the delivery of your software. One drawback is limited options for module dependencies, which this post will address.
I have been coding since 6th grade, circa 1986, professionally (i.e. college graduate) since 1998 when I graduated from the University of Nebraska-Lincoln. Most of my career has been in web applications using JEE. I work the entire stack from user interface to database. I especially like solving application security and high availability problems.