This Post will talk about our custom Building System used to create the PloobsEngine MSI Package. Lots of time ago, when the engine code and the project number were smaller, we could afford to build our packages by hand, but when things got bigger and bigger the necessity for something fully automatizated became obvious.
The main funcionalities of our Building System are:
- Download Latest/Specific version of your repo (Mercurial)
- Compile all Projects, Tests and Demos
- Run Some Tests
- Compile Documentation
- Create and compress the Package (generate the Msi file)
- Upload to a Server (ftp)
- Notify People (send emails)
Here we will describe with details how each part was implemented, showing some parts of the used scripts. (Everything is hosted in a Windows 7 machine).
Download Latest/Specific version of your repo.
As said, We used Mercurial for source code management. To download the repo and update to the latest version we used the following script.
@echo off echo Downloading Ploobs HG Repo cd / if exist %RepoPath%/port-ploobsengine goto NEXT md %RepoPath%\ cd %RepoPath%\ call hg clone https://email@example.com/p/port-ploobsengine/ if %errorlevel% 1 ( echo Failure Reason Given is %errorlevel% chdir /d %OLDDIR% exit /b %errorlevel% ) cd / :NEXT cd %RepoPath%port-ploobsengine call hg pull call hg update --clean cd /
The script is very simple, it just creates the repo folder and clone the repo (if it does not exists yet) and then pull and updates its contents to the latest remote version. (the bin folder of the TortoiseHg must be on the Windows Path. If you use SVN or git or …. just adapt the script).
To those that are curious, our public repo is: http://code.google.com/p/port-ploobsengine/
Compile all Projects, Tests and Demos
We have lots of different types of projects in PloobsEngine, then we need to use different tools to build each of them. The following script shows how to deal with these situations:
First of all: Set some Environment Variables (MSBuild Path, XNA PATH, .Net Framework Path …)
SET PATH=%*;%XNAGSv4%Tools;%XNAGSShared%XnaPack;%XNAGSShared%Device Center;C:\Windows\Microsoft.NET\Framework\v4.0.30319\;C:\Windows\Microsoft.NET\Framework\v3.5\;C:\Windows\Microsoft.NET\Framework\v2.0.50727;%PATH%
Using Msbuild for content pipeline and main XNA project
The content pipeline / XNA project are just “regular” Visual Studio projects, to build them we just used the following lines (adapt to your needs)
msbuild %RepoPath%\PloobsEngine\PloobsEngineContent\PloobsEngineContent.contentproj /p:XNAContentPipelineTargetPlatform=Windows;XNAContentPipelineTargetProfile=HiDef;TargetFrameworkVersion=v4.0;Configuration=Release msbuild %RepoPath%\PloobsEngine\PloobsEngine\PloobsEngine.csproj /p:Configuration=Release
Using the Visual Studion IDE compiling ability directely (msbuild cant compile Visual Studio Installer projects !!!)
Some projects (like those that ends with .vdproj) cant be built using MSBuild, then we call directely the Visual studio line comand IDE utility.
echo Compiling Ploobs Updater "%VS100COMNTOOLS%..\IDE\devenv.com" /build Release %RepoPath%\PROJEC_PATH\PloobsInstallerSetup.vdproj rd /s/q FOLDER md FOLDER copy %RepoPath%\PROJECT_PATH\*.* FOLDER /Y
Just building and copying the generated contents to a folder (deleting if it already exists)
Using Msbuild to Compile/Clean Full Solutions
echo Copy Others Demos rd /s/q SourceCodeDemos md SourceCodeDemos msbuild %RepoPath%\PloobsUpdater\PloobsUpdater.sln msbuild %RepoPath%\AdvancedDemos\AdvancedDemos.sln /t:Clean msbuild %RepoPath%\IntroductionDemo\IntroductionDemo4.0\IntroductionDemo4.0.sln /t:Clean msbuild %RepoPath%\ReachDemos\ReachDemos.sln /t:Clean msbuild %RepoPath%\PhoneSilverLightDemo\GraphicsApp1.sln /t:Clean
Just building the PloobsUpdater.sln Solution and cleaning some others (removing the Bin and Obj folders -> removing previous builds contents, we dont need to send then to the clients !)
Calling Some Custom Scripts
PloobsEngine depends of LOOTS of Dlls, for the end user is easier to have just one with everything inside of it, the following code shows how this can be achieved (using the Microsoft ILMerge Program)
ilmerge /out:PloobsEngineDebug.dll SharedOsiris.dll PloobsEngine.dll FarseerPhysicsXNA.dll Lidgren.Network.dll Library.dll TomShane.Neoforce.Controls.dll BEPUphysics.dll XNAnimation.dll DPSF.dll /lib:"C:\Program Files (x86)\Microsoft XNA\XNA Game Studio\v4.0\References\Windows\x86" /targetplatform:v4,"C:\Windows\Microsoft.NET\Framework\v4.0.30319" /keyfile:XNAnimation.snk /xmldocs /ver:0.0.0.1 /log
For more informations about the arguments, look at the ILMerge Documentation. In this line we just merged Lots of dlls (and the xml docs of each one) and signed the generated assembly with a keyfile.
Running Some Tests
To make sure that we are delivering something that works, our building system also implements some custom unity and Visual Tests. The following script calls our Test Module and checks its answer. (if somethings wrong happens, errors are directed to us !)
echo Executing Visual Tests cd AdvDemosBin call AdvancedDemos.exe VisualUnitTests echo Return Code is %errorlevel% if %errorlevel% == 0 goto LB echo Tests Failed, Check the Logs set erro=1 #### Pack the logs and mail the programmers =P .... exit /b -1 :LB del *.log cd ..
Compiling the Documentation
We use the marvelous SandCastle project to create our documentation.
Sandcastle produces accurate, MSDN style, comprehensive documentation by reflecting over the source assemblies and optionally integrating XML Documentation Comments. It also let us to create some custom pages in order to make a full professional like manual.
The amazing thing is that SandCastle compilation can be triggered using msbuild. (the output is a .chm file), the following script shows how:
if %NODOC% == "FALSE" goto ND echo Compiling Ploobs Documentation msbuild %RepoPath%\Documentation\EngineHelper.shfbproj rd /s/q Help md Help copy "%RepoPath%\Documentation\Ploobs Engine Help.chm" Help /Y :ND
Creating the Installer (using the marvelous InstallShield Visual Studio Extension)
Each piece are put together with the Install Shield Installer (ends up up a MSI package). The building process can also be triggered using msbuild !!!
echo Creating the Installer rd /s/q PloobsEngine md PloobsEngine rd /s/q %RepoPath%\PloobsInstaller\PloobsInstaller\Express msbuild %RepoPath%\PloobsInstaller\PloobsInstaller.isproj /p:Configuration=Release;Build=Complete;BuildCompressed=true;BuildSetupExe=true xcopy %RepoPath%\PloobsInstaller\PloobsInstaller\Express\Release\DiskImages\DISK1\*.* PloobsEngine /Y /e /i /h
We also run some custom scripts to set the right minor/minor version of the release.
Compressing with Rar (using the classic Winrar)
After the package generation, we need to compress it to a single file (not a requirement in most cases). We just used the classic Winrar for this task.
set PATH="C:\Program Files (x86)\WinRAR";%PATH% cd PloobsEngine rd /s/q Package md Package cd .. rar a -r "PloobsEngine/Package/PloobsEngine.rar" "PloobsEngine/*" copy %DeployPath%\Changes.pdf %DeployPath%\PloobsEngine\Package\ /Y
Uploading to FTP server (generated from the previous pass)
Then we make a simple script to connect to our FTP server to upload the rar file to the righ folder
@echo off echo Uploading Installer to the Main Server echo user USER> ftpcmd.dat echo PASSWORD>> ftpcmd.dat echo cd Web/Updater>> ftpcmd.dat echo mkdir teste>> ftpcmd.dat echo cd teste>> ftpcmd.dat echo bin>> ftpcmd.dat echo mput piece.bat>> ftpcmd.dat echo y>> ftpcmd.dat echo quit>> ftpcmd.dat ftp -n -s:ftpcmd.dat FTP_URL del ftpcmd.dat
The script just generates a dummy conf file (to be able to log in the remote ftp server) and uses it as a parameter to the FTP command.
You can download our packages here: http://www.ploobs.com.br/Updater/0.4.4/PloobsEngine.rar
Sending some Emails and notifications
The last step is notifying everyone involved about the new release. For this we just send some email to the righ people (We made a custom .exe to perform this task, unfortunaly the code is not avaliable but this is a very simple thing to build)
We also have some things to adjust the version number of the releases that were not shown here.
The whole proccess takes more than 4 hours to be completed and is fully automatized. (We remotely calls it and waits for the email containing the Download Link !!!)
There are lots of good build systems avaliable like Team City. I always encourage people to use them. Exceptionaly in our case, i prefered to do all by hand because of the large amount of customization we needed.