Builder Simple Example

From ControlTier

Requires Version 3.4.3 (help?)

This Builder Simple Example shows how to use an object of the generic Builder Type to coordinate a build process for a simple web application.

This example shows you how to do the following things:

1. Configure a Builder object to:
   • Use a CVS repository for code checkout
   • Use an Ant build script that is part of the checked out sources
   • Use a predefined buildstamp number for the build artifact
   • Use a manually specified buildstamp number for the build artifact
   • Import the produced build artifact to the ControlTier repository as a Package object
2. Use the Builder Build workflow, and its component commands
3. use the project.xml resource model format to define a Builder object and its Settings

This example automates a very simple build process for a web application project named "builder-simple". The web application is a simple JSP based app, that will be built using Ant. The Ant build process will create a ".war" file, and the Builder will be configured to look for that file after the build and to upload it to the repository along with a new Package definition. All of the source code for the web application will be stored in a local CVS repository, which will be used to show how the Builder can do the SCM checkout.

The buildstamp for the build will be specified with Setting objects, defining each component of the buildstamp string, which uses the format: "tag.major.minor.release". The actual setting values will define the value to be: "trunk.1.2.3". The build will also be shown using a user-specified buildstamp when the Build workflow is executed.

The built ".war" file will be uploaded to the ControlTier package repository after the build, and registered as a Package object of type war.

This example doesn't cover deploying the webapp once it has been built. For that purpose see the Service Package Deployment Example.

Since this example uses the base Builder type, the build mechanism is a simple Ant build script without any customization. More specific Builder types are available, such as the AntBuilder which allows more customization of the Ant build, the MavenBuilder which builds Maven 2 projects, and more.

Contents 1 Dependencies 2 Building the Example 3 The web application source 4 Create a CVS repository 5 The build.xml 6 Noteworthy settings 7 Running the Example 7.1 Examine the Builder properties 7.2 The Build workflow command 7.3 Automatic buildstamp generation 7.4 User specified buildstamp 8 Walking through the Build output messages 8.1 Step 1: scmCheckout 8.2 Step 2: setBuildstamp 8.3 Step 3: runBuildscript 8.4 Step 4: repoImport 9 How it Works 9.1 The Setting definitions 9.1.1 SCM info Settings 9.1.2 Version Settings 9.1.3 Build script Settings 9.1.4 Package Import info Settings 9.2 The Builder Deployment 10 Related Topics

Dependencies

This example has these dependencies.

• ControlTier — 3.4.3
  • Meet the prerequisites and install ControlTier according to the Installing ControlTier instructions.
• Unix
  • This example is not compatible with Windows. It is compatible with Linux, Mac OS X and most unix variants.
• CVS
  • If the CVS binary is not available on your machine, download and install the CVS package. Several binaries for different OS and architectures, as well as source code can be found at www.nongnu.org/cvs. Once installed, be sure to add it to your PATH.

Building the Example

Follow the instructions in this section to setup the example code into your environment.

Note: Don't worry about what these commands do, as they just bootstrap the example code to work in your environment and to pre-load the resource model for you. (For complete detail about how to use the Examples see Using the Examples):

Execute these bootstrap steps:

1. cd $CTIER_ROOT/examples/builder-simple

   • Navigate to the examples/builder-simple directory under your $CTIER_ROOT directory.
2. Edit the file: templates/defaults.xml
   • If you use a different node name that 'localhost', modify the file templates/defaults.xml to set the value.

3. ctl -p demo -m ProjectBuilder -c Register -- -xml projectbuilder.xml -install

   • This loads a ProjectBuilder object definition into the ControlTier Server.

4. ctl -p demo -t ProjectBuilder -o builder-simple -c Build

   • Builds a working example based on template files and your working environment. Later see Further Customization

You are now ready to run the example.

The web application source

You are now ready to configure the CVS repository.

The source for a simple web application is available in the directory $CTIER_ROOT/examples/builder-simple/content/simple

The source files implement a trivial webapp. There will be two main source files: a JSP and a web.xml.

Create a CVS repository

The source files will be maintained in a local CVS repository. Run the following commands to create the CVS instance and import the sources:

cd $CTIER_ROOT/examples/builder-simple
mkdir -p demo
export CVSROOT=`pwd`/demo/cvsroot; # local CVS repo path
cvs -d $CVSROOT init;                # create the repository
cd content/simple/src;                 # change directory and import the sources
cvs import -m "Importing Simple example" src tutorial start; 

The build.xml

An Ant build.xml file is present at $CTIER_ROOT/examples/builder-simple/content/simple/src/build.xml that will build and package the release artifact.

Notice the embedded properties ${opts.basedir}, ${opts.targetdir} and ${opts.buildstamp}. These properties will be passed in via the Builder's "runBuildscript" command and be replaced with the values of the configured settings.

Noteworthy settings

Reviewing the steps taken in preparing the simple application's source tree a few noteworthy details should be highlighted:

Setting	Value	Description
SCM connection	$CTIER_ROOT/examples/builder-simple/demo/cvsroot	path to the source in the source code repository. (used by scmCheckout)
SCM module	src	module to checkout of the repository (used by scmCheckout)
source base directory	$CTIER_ROOT/examples/builder-simple/demo/builder-simple	directory where checked out source module resides (used by scmCheckout and runBuildScript)
build file path	$CTIER_ROOT/examples/builder-simple/demo/builder-simple/src/build.xml	the build file to invoke (used by runBuildScript)

Each of these values is configured for the Builder in the default-object.xml that has been generated. In a later section we will walk through the content of the file.

Running the Example

You can run any of the Builder commands via CTL like so:

ctl -p demo -t Builder -o builder-simple -c command-name

If you run the command without the "-c command-name" parameter you will see a listing of commands.

Examine the Builder properties

With the Builder configuration loaded you can examine the Builder configuration attributes via CTL.

Run the Get-Properties CTL command to look at the information about the new object:

ctl -p demo -t Builder -o builder-simple -c Get-Properties -- -print

[MULTI_LINE]
# builder-simple [Builder] #

A Builder to build and package the builder-simple app

## Attributes ##

*  basedir: "$CTIER_ROOT/examples/builder-simple/demo/builder-simple"
*  buildFile: "$CTIER_ROOT/examples/builder-simple/demo/builder-simple/src/build.xml"
*  buildTarget: "all"
*  buildstamp: "0"
*  defaultAllowMultiplePackageMatches: "false"
*  defaultDeploymentType: "Builder"
*  defaultFailIfPackageNotReplaced: "true"
*  defaultPackageName: ".*"
*  defaultPackageProximity: "1"
*  defaultPackageType: "[^\.]*"
*  dirlist: "bin,logs,var"
*  errorNotificationRecipient: "${framework.admin.tolist}"
*  importMax: "1"
*  importMin: "1"
*  packageBuildtimePattern: "yyyymmddhhmmss"
*  packageExtension: "war"
*  packageFilebase: "builder-simple"
*  packageInstallRank: "0"
*  packageInstallroot: "${entity.attribute.jetty_basedir}/webapps/builder-simple"
*  packagePurgeRegex: "^(?!^0$).*$"
*  packageSeparator: "-"
*  packageType: "war"
*  packageVersion: "${opts.buildstamp}"
*  requireBuildstamp: "false"
*  scmBinding: "cvs"
*  scmConnection: "$CTIER_ROOT/examples/builder-simple/demo/cvsroot"
*  scmLabel: ""
*  scmModule: "src"
*  scmQuiet: "false"
*  scmUser: "greg"
*  successNotificationRecipient: "${framework.admin.tolist}"
*  targetdir: "$CTIER_ROOT/examples/builder-simple/demo/builder-simple/target"
*  versionMajor: "1"
*  versionMinor: "2"
*  versionRelease: "3"
*  versionTag: "trunk"

.
.
.
- - -
[/MULTI_LINE]

If you go to Workbench, and navigate to the Builder List, you will see the "builder-simple" Builder object.

The Build workflow command

To run the whole build cycle, use the Builder's Build workflow command. The Build command will execute the sequence of commands that comprise each step of the build life cycle.

The "Build" command takes an optional parameter, "-buildstamp". The "buildstamp" represents the build identifier is also used during package registration in the repository import step. If no -buildstamp parameter is specified, one will be automatically generated.

If you are curious what commands will run in the Build workflow view the "process flow view" in Workbench. Navigate to the "builder-simple" Builder object in Workbench, select it and then click the "Commands" tab. Scroll to the "Build" command and press the icon to the right.

The image shows the sequence of commands Build will run and on which Node the command can be run.

Automatic buildstamp generation

Run the Build command without any options and the Build command will generate a buildstamp to identify the produced artifact.

execute:

ctl -p demo -t Builder -o builder-simple -c Build

The output from this command is fairly long, and is listed in #Walking through the Build output messages, but note that these steps occur:

• Build begins by running the scmCheckout command which runs the CVS checkout of simple's src directory.
• Next, the setBuildstamp command generates and stores a buildstamp used in later steps of the workflow
• The runBuildScript command is then run to invoke the build procedure.
• Finally, the repoImport command looks in the target directory for a matching artifact and then puts it into the repository, recording information about the package in the model.

After the command completes, go to the Package Manager in Workbench and list all the packages. You should see a package named "builder-simple-trunk.1.2.3.war".

User specified buildstamp

You can also run the Build command and pass in an arbitrary buildstamp value. This lets a user define their own build identifier via the command option.

To use the buildstamp "Mar15" as the buildstamp value, add the -buildstamp option.

execute:

ctl -p demo -t Builder -o builder-simple -c Build -- -buildstamp Mar15
.
.
. 
<output omitted>

Notice how the "Mar15" buildstamp is used throughout the steps of the workflow.

Go back to the Package Manager package list and you should now see a packaged named "builder-simple-Mar15.war".

Walking through the Build output messages

It's helpful to walk through the output of the Build workflow to understand what happened at each step. Each step in the workflow is demarcated with a begin/end message.

Step 1: scmCheckout

The scmCheckout command invokes the "cvs checkout" command passing in the needed parameters. You can see these in the second line as "scmCheckout parameters":

begin workflow command (1/4) -> "scmCheckout " ...
scmCheckout parameters: {basedir="$CTIER_ROOT/examples/builder-simple/demo/builder-simple", \
    connection="$CTIER_ROOT/examples/builder-simple/demo/cvsroot", module="src", label="", scmcommand="checkout" }
Created dir: $CTIER_ROOT/examples/builder-simple/demo/builder-simple
cvs checkout: Updating src
U src/build.xml
cvs checkout: Updating src/WEB-INF
U src/WEB-INF/web.xml
cvs checkout: Updating src/jsp
U src/jsp/index.jsp
[command.timer.Builder.scmCheckout: 1.128 sec]
end workflow command (1/4) -> "scmCheckout "

Because this Builder's specified "cvs" for the BuilderScmBinding, the cvs client was invoked. The cvs client implementation then formats the correct set of arguments for the cvs command.

Step 2: setBuildstamp

The setBuildstamp command conditionally generates a buildstamp if one is not supplied. It also stores it so the last used buildstamp is known.

begin workflow command (2/4) -> "setBuildstamp " ...
CVS does not support automatic change number generation, skipping ...
Automatically set buildstamp to "trunk.1.2.3"
[command.timer.Builder.setBuildstamp: 0.686 sec]
end workflow command (2/4) -> "setBuildstamp "

Notice in the third line, "trunk.1.2.3" is the generated buildstamp.

Step 3: runBuildscript

The runBuildscript command is responsible for running the build tool, script and target. The second line shows the parameters used.

begin workflow command (3/4) -> "runBuildScript " ...
runBuildScript parameters: {basedir="$CTIER_ROOT/examples/builder-simple/demo/builder-simple", \
    targetdir="$CTIER_ROOT/examples/builder-simple/demo/builder-simple/target", buildstamp="trunk.1.2.3", \
    buildfile="$CTIER_ROOT/examples/builder-simple/demo/builder-simple/src/build.xml", target="all" }
Created dir: $CTIER_ROOT/examples/builder-simple/demo/builder-simple/target
Building war: $CTIER_ROOT/examples/builder-simple/demo/builder-simple/target/builder-simple-trunk.1.2.3.war
[command.timer.Builder.runBuildScript: 8.491 sec]
end workflow command (3/4) -> "runBuildScript "

Note also, any of the "builder-simple" Builder's property data can be referenced within the build.xml file because the Ant environment is passed into the Ant build.

Step 4: repoImport

The last step in the generic Build workflow is to call repoImport. The repoImport command looks for artifacts to upload to the repository. It looks for files in the "targetdir" recursively, searching for files that match the configured filebase pattern and file extension.

begin workflow command (4/4) -> "repoImport " ...
Created dir: $CTIER_ROOT/ctl/depots/demo/deployments/Builder/builder-simple/tmp/Builder-repoImport-20090416121609
Condition: isProcessableWithoutExistingPropfile result: true
opts.propfile set to: autogenerated-builder.properties
Creating new property file: \
    $CTIER_ROOT/ctl/depots/demo/deployments/Builder/builder-simple/tmp/Builder-repoImport-20090416121609/import-count.properties
processing files in directory: '$CTIER_ROOT/examples/builder-simple/demo/builder-simple/target' \
    matching: '(builder-simple)(?:-trunk.1.2.3)?\.(war)$' ...
processing matched file: builder-simple-trunk.1.2.3.war
Auto-generated builder properties file created: $CTIER_ROOT/examples/builder-simple/demo/builder-simple/target/autogenerated-builder.properties...
Copying 1 file to $CTIER_ROOT/examples/builder-simple/demo/builder-simple/target
Updating property file: $CTIER_ROOT/ctl/depots/demo/deployments/Builder/builder-simple/tmp/Builder-repoImport-20090416121609/import-count.properties
Copying 1 file to $CTIER_ROOT/ctl/depots/demo/modules/Builder/commands
Getting: http://localhost:8080/jackrabbit/repository/workbench/demo/publish/modules/war-head.jar
To: $CTIER_ROOT/ctl/var/tmp/downloads/demo/war-head.jar
Created dir: $CTIER_ROOT/ctl/depots/demo/modules/war
Expanding: $CTIER_ROOT/ctl/var/tmp/downloads/demo/war-head.jar into $CTIER_ROOT/ctl/depots/demo/modules/war
Attempting to get Package-head.jar ...
Getting: http://localhost:8080/jackrabbit/repository/workbench/demo/publish/modules/Package-head.jar
To: $CTIER_ROOT/ctl/var/tmp/downloads/demo/Package-head.jar
Expanding: $CTIER_ROOT/ctl/var/tmp/downloads/demo/Package-head.jar into $CTIER_ROOT/ctl/depots/demo/modules/Package
uploading file: $CTIER_ROOT/examples/builder-simple/demo/builder-simple/target/builder-simple-trunk.1.2.3.war to: \
    http://localhost:8080/jackrabbit/repository/workbench/pkgs/demo/war/wars/builder-simple-trunk.1.2.3.war...
Uploading to: http://localhost:8080/jackrabbit/repository/workbench/pkgs/demo/war/wars/builder-simple-trunk.1.2.3.war
Uploading: builder-simple-trunk.1.2.3.war
Puted 1 file to http://localhost:8080/jackrabbit/repository/workbench/pkgs/demo/war/wars/builder-simple-trunk.1.2.3.war
Creating new property file: \
    $CTIER_ROOT/ctl/depots/demo/deployments/Builder/builder-simple/tmp/Builder-repoImport-20090416121609/import-list.properties
Deleting: $CTIER_ROOT/ctl/depots/demo/modules/Builder/commands/entity1160301077.properties
Current count: 1
Batch request performed successfully.
Current count: 1
[command.timer.Builder.repoImport: 42.437 sec]
end workflow command (4/4) -> "repoImport "

Notice on line 6 of the output processing files in directory: '$CTIER_ROOT/examples/builder-simple/demo/builder-simple/target' \ matching: '(builder-simple)(?:-trunk.1.2.3)?\.(war)$' ...

This line shows that repoImport begins looking for files in the target dir "$CTIER_ROOT/examples/builder-simple/demo/builder-simple/target". It looks for files that match the pattern '(builder-simple)(?:-trunk.1.2.3)?\.(war)$'. Breaking down that regular expression into the parameters:

• filebase + buildstamp + extension.

How it Works

The Builder is defined in a project XML file generated into the $CTIER_ROOT/examples/builder-simple. directory. This section walks through the XML definition files used to define the Setting and Builder objects

The Setting definitions

The Settings define attributes (see: Builder Settings and Attributes) that the various commands of the Builder type use. For our simple Builder, we configure it to use the paths mentioned above in the #Noteworthy settings section.

The specific commands of the Build workflow use the Attribute Values defined below, and their corresponding Setting subtypes are shown:

• scmCheckout
  • scmBinding - BuilderScmBinding
  • scmConnection - BuilderScmConnection
  • scmModule - BuilderScmModule
• setBuildstamp
  • versionMajor - BuilderVersionMajor
  • versionMinor - BuilderVersionMinor
  • versionRelease - BuilderVersionRelease
  • versionTag - BuilderVersionTag
  • buildstamp - BuilderBuildstamp
• runBuildscript
  • buildFile - BuilderBuildFile
  • buildTarget - BuilderBuildTarget
• repoImport
  • packageType - BuilderPackageType
  • packageExtension - BuilderPackageExtension
  • packageFilebase - BuilderPackageFilebase
  • packageInstallroot - BuilderPackageInstallroot

SCM info Settings

Three Settings are used to configure the SCM info about the CVS instance that was just setup, and are used by the scmCheckout command.

• The type of SCM system used is determined by the scmBinding attribute. In this example we use CVS, so the value is "cvs".
• The SCM connection string is determined by the scmConnection attribute.
• The source code module name is set by the scmModule attribute.

Here are those setting definitions in XML:

    <setting type="BuilderScmModule" name="builder-simple" description="The builder-simple app surce module name" settingValue="src"/>
    <setting type="BuilderScmBinding" name="builder-simple" description="CVS binding for source code" settingValue="cvs"/>
    <setting type="BuilderScmConnection" name="builder-simple" description="Path to the CVS root" 
        settingValue="$CTIER_ROOT/examples/builder-simple/demo/cvsroot"/>

Version Settings

After checkout, the Builder needs to know a buildstamp to use for the produced artifact.

• The setBuildstamp command generates a buildstamp
• Provides an open-ended strategy to define the packaged artifacts version information

These attributes are used by the setBuildstamp command:

1. versionTag - a textual tag to prepend to the buildstamp, "trunk" in this example.
2. versionMajor - the Major version number, "1" for this example
3. versionMinor - the Minor version number, "2" for this example
4. versionRelease - the release number, "3" for this example
5. buildstamp - the value of the generated buildstamp, "0" used as a place-holder until the first buildstamp is generated

The automatic-buildstamp generation feature of Builder will combine the first 4 of these attributes into a buildstamp, in the pattern:

tag.major.minor.release

Note that these are all optional when defining your own Builder.

Here they are as setting definitions in XML:

    <setting type="BuilderVersionTag" name="builder-simple" description="The initial builder-simple app build tag" settingValue="trunk"/>
    <setting type="BuilderVersionMajor" name="builder-simple" description="The builder-simple app major version number" settingValue="1"/>
    <setting type="BuilderVersionMinor" name="builder-simple" description="The builder-simple app minor version number" settingValue="2"/>
    <setting type="BuilderVersionRelease" name="builder-simple" description="The builder-simple app release version number" settingValue="3"/>
    <setting type="BuilderBuildstamp" name="builder-simple" description="The initial builder-simple app buildstamp value" settingValue="0"/>

Build script Settings

The third phase of the Build life cycle is to invoke the build procedure.

• The "runBuildscript" invokes the configured build tool and build script with specified target.
• Execution relative to "basedir"
• Artifacts generated to "targetdir"
• By default can execute an Ant build file
• You might want to override it to invoke your build procedure
  • Check for existing types before overriding it
  • For example these other Builder types exist: AntBuilder, MavenBuilder, BatBuilder, RpmBuilder, ZipBuilder, etc

The attributes used by the runBuildscript command are:

1. buildFile - the path to the buildfile to use by the Ant build system, we use the path to the build.xml file in the checked-out sources dir.
2. buildTarget - the target to pass to the Ant build process, in this example "all"

Here they are as setting definitions in XML:

    <setting type="BuilderBuildFile" name="builder-simple" description="Simple app Ant build file: build.xml" 
        settingValue="$CTIER_ROOT/examples/builder-simple/demo/builder-simple/src/build.xml"/>
    <setting type="BuilderBuildTarget" name="builder-simple" description="Simple app Ant build target: all" settingValue="all"/>

Package Import info Settings

The last step in the Build life cycle is to find the produced artifacts and import them to the repository.

• The repoImport command searches from the targetdir directory and imports matching files.
• Used as metadata for package registration

The attributes used by repoImport are:

1. packageType - the name of a ControlTier Package subtype representing the type of the imported file, in this case "war"
2. packageExtension - a string to match the file extension of the build artifact. Our build produces a ".war" file, so we set it to "war"
3. packageFilebase - the base name of the build artifact. our build.xml produces a "builder-simple-VERSION.war", so we set it to "builder-simple"
4. packageInstallroot - A path to set as the package-install-root property of the imported Package object. We configure it for this example to specify a webapps dir in a hypothetical Jetty installation. "${entity.attribute.jetty_basedir}/webapps/builder-simple".

Here they are as setting definitions in XML:

    <setting type="BuilderPackageType" name="builder-simple" description="Type name for Builder produced Package" settingValue="war"/>
    <setting type="BuilderPackageFilebase" name="builder-simple" description="File base name of the build artifact" settingValue="builder-simple"/>
    <setting type="BuilderPackageExtension" name="builder-simple" description="File extension of the build artifact" settingValue="war"/>
    <setting type="BuilderPackageInstallroot" name="builder-simple" description="Installation directory to set for the Package when deployed" 
        settingValue="${entity.attribute.jetty_basedir}/webapps/builder-simple"/>

These are the last of the settings.

The Builder Deployment

The Deployment definition for the Builder requires noting these details:

1. The name and type: We're using the generic Builder type. Call the instance "builder-simple" to correspond with the app name.
2. The basedir directory: This is the directory where the sources will checked out
3. The installroot directory: This is where targets are written
4. Node referrer: This is the Node(s) where the Builder will be assigned.
5. The type and name of each of the Setting objects we want to assign to the Builder

The complete file is at the path: $CTIER_ROOT/examples/builder-simple/default-object.xml

This snippet shows the Builder deployment definition:

    <!--
        Builder deployment definition
    -->
    <deployment 
        type="Builder"
        name="builder-simple" 
        description="A Builder to build and package the builder-simple app"        
        basedir="$CTIER_ROOT/examples/builder-simple/demo/builder-simple"
        installRoot="$CTIER_ROOT/examples/builder-simple/demo/builder-simple/target"    
        >
        <referrers replace="false">
            <!-- add the Node as a parent dependency (referrer) -->
            <resource type="Node" name="..."/>
        </referrers>
        <resources replace="true">
            <!-- add all of the settings as child dependencies (resources) -->
            <resource type="BuilderScmModule"           name="builder-simple"/>
            <resource type="BuilderScmBinding"          name="builder-simple"/>
            <resource type="BuilderScmConnection"       name="builder-simple"/>
            <resource type="BuilderVersionTag"          name="builder-simple"/>
            <resource type="BuilderVersionMajor"        name="builder-simple"/>
            <resource type="BuilderVersionMinor"        name="builder-simple"/>
            <resource type="BuilderVersionRelease"      name="builder-simple"/>
            <resource type="BuilderBuildFile"           name="builder-simple"/>
            <resource type="BuilderBuildTarget"         name="builder-simple"/>
            <resource type="BuilderBuildstamp"          name="builder-simple"/>
            <resource type="BuilderPackageType"         name="builder-simple"/>
            <resource type="BuilderPackageFilebase"     name="builder-simple"/>
            <resource type="BuilderPackageExtension"    name="builder-simple"/>
            <resource type="BuilderPackageInstallroot"  name="builder-simple"/>  
        </resources>
    </deployment>

The XML file is uses the project.xml format and a "deployment" tag to define the Builder deployment. The deployment tag has attributes for the items described in the numbered bullet list.

The Node reference is done via the "referrers" and "resource" tag.

Each of the Settings we have already defined is added as a child dependency under the "resources" section.

Related Topics

• Continuous Deployment Example with Tomcat shows how to automatically build and deploy a WAR to a Tomcat server after a code-change is performed.

v • d • e Examples Scripting Replace your SSH loop script  · Scripted appserver war deployment example Services Mock Unix Service Example  · Mock Site Example  · Service Package Deployment Example  · Windows service  · Windows site  · Windows deployment Builders Builder Simple Example  · Continuous Deployment Example with Tomcat J2EE Dukes Bank J2EE examples