Source Control Directory Structure

Here is a directory structure that I use for all of my projects. I figured, if nothing else, it would be a great discussion point.

Directory Structure

The directory
structure for a project should be set up in the suggested manner:




Definitions:

1. Releaseable Project - A project that has its own
   release schedule. All folders
   underneath the project/trunk folder should be on the same release
   schedule. If an item is on a different
   release schedule it should be analyzed for a possible move to its own
   project folder.

This accomplishes
several tasks:

1. When the Trunk is branched not only is the source
   branched, but all items related to the creation of the application will
   be branched as well.
2. Directory clutter that often
   results when there are several groups trying to use the same space will
   be reduced
1. Build files will be placed in the build directory
2. Custom or external tools
   will be placed in the tools directory
3. Documentation can have its
   own space
3. The extra directories (Database, Tools, Documentation,
   etc…) are optional and only needed if the project requires them. Build and Src should be required
   directories.

Build Directory

The build directory
is used by the build manager to keep the source directory from being
cluttered. This will allow build
managers their own 'space'.

Src Directory

The Source
directory (src) contains all items related to the applications source
code. This is relatively loosely
defined as it can contain custom tool code that may not be shipped with the
application, documentation, etc… This
decision is to be made by the development team.

TeamBuild 2008 Override(able) properties

I finally got tired of having to look these properties up and decided to place them in here (what makes looking on the web easier than looking in the actual file, I don't know, but maybe this will help somebody else as well).

Properties

MSTestRefPath - Path to Microsoft.VisualStudio.QualityTools.MSBuildTasks.dll. Overridable so that the 8.0 version can be used for test assemblies compiled against the 8.0 unit test framework.

V8TestToolsTask - Set this property to true to use the 8.0 TestToolsTask.

IncrementalGet - Set this property to true to do an incremental get - this will override the CleanCompilationOutputOnly, SkipInitializeWorkspace, and ForceGet properties.

IncrementalBuild - Set this property to true to do an incremental build - this will override the SkipClean, SkipInitializeWorkspace, and ForceGet properties.

BuildProjectFolderPath - If BuildProjectFolderPath is not specified (typically in a Desktop build), assume local paths and set to MSBuildProjectDirectory.

SkipClean - Set this property to true to skip the CoreClean target.

SkipLabel - Set this property to true to skip the CoreLabel target.

SkipPostBuild - Set this property to true to skip the PostBuild target.

SkipGetChangesetsAndUpdateWorkItems - Set this property to skip the CoreGetChangesetsAndUpdateWorkItems target, which calls the GenCheckinNotesUpdateWorkItems task.

SkipDropBuild - Set this property to true to skip the CoreDropBuild target.

SkipWorkItemCreation - Set this property to true to skip the CoreCreateWorkItem target.

BuildConfigurationsInParallel - Set this property to true to enable building configurations in parallel.

BuildSolutionsInParallel - Set this property to true to enable building solutions in parallel.

SkipInvalidConfigurations - Set this property to false to generate an error when an invalid configuration is encountered.

StopOnFirstFailure - Set this property to true to stop Cleaning, Compiling, and/or Testing on the first failure encountered.

WorkspaceName - The name of the workspace that will be used for getting sources.
Note: Workspace name can be up to 64 characters long - after that it will be truncated.

CreateWorkspaceTaskComment - The comment used for workspace creation.

GetOverwrite - Set this value to true to enable overwriting of writable files without a force get.

RecursiveGet - Set this to false to do only a top-level get.

GetVersion - The VersionSpec to be used by the Get task.

GetFileSpec - The FileSpec to be used by the Get task. When empty, all top-level folders in the workspace are used.

GetPopulateOutput - Set this to true to populate the Gets, Replaces, and Deletes item group outputs of the Get task.

LabelRecursive - Set this to false to do only a top-level label.

LabelChild - Set this to Merge or Replace to control the treatment of unchanged items.

LabelComment - The comment used by the Label task.

LabelName - The label name used by the Label task. By default the label name is set to $(BuildNumber) in the InitializeEndToEndIteration target.

LabelFiles - The FileSpec to be used by the Label task Default:"$/".

LabelScope - The scope to be used by the Label task Default:"$/$(TeamProject).

VCBuildAdditionalLibPaths - The AdditionalLibPaths property of the VCBuild task used to compile VC++ projectstd.

VCBuildAdditionalOptions - The AdditionalOptions property of the VCBuild task used to compile VC++ projectstd.

VCBuildToolPath - The ToolPath property of the VCBuild task used to compile VC++ projects.

VCBuildUseEnvironment - The UseEnvironment property of the VCBuild task used to compile VC++ projects.

VCOverridesOpen - The beginning of the vsprops file used to override VCBuild properties.

VCOverridesClose - The end of the vsprops file used to override VCBuild properties.

UpdateAssociatedWorkItemsOnBuildBreak - Set this to true to update associated work items even on a build break.

CustomizableOutDir - Set this to true to allow OutDir customization.

CustomizablePublishDir - Set this to true to allow PublishDir customization.

OutDir - Set OutDir so that it is always available within TfsBuild.proj for backwards compatibility.

PublishDir - Set PublishDir so that it is always available within TfsBuild.proj for backwards compatibility.

TeamBuild 2008 - Passing properties to Solutions

Our build process requires that we label our builds according to the version that we are building, i.e. Assemblies will be stamped with 1.0.2.1 so the build label should be BuildDefinition_1.0.2.1-BuildCount. I won't detail the method to create the custom build number as that has already been done very nicely by Martin Woodward - you can see that here.

What hasn't been explained, for which I spent a few hours (okay days) trying to figure out was how to pass that version to our solutions.

Here's our process:
I created a custom build task that stored the version by build definition and incremented that version on each successful build (the script passes the build definition to the task, the task sets the output properties, if the last build was not successful, the version would not have been incremented instead the BuildCount would be incremented showing how many times that particular version was attempted). I then passed the output from the task to the SolutionToBuild list. The initial try left me with my default of 0.0.0.0 (discussion on whether that was right to default it to that can be held at another time). I thought my default property was overriding my output from the task. That was not the case. I finally found that the property was not being passed appropriately to the Item list.

I finally broke down and consulted the expert (at least the one I knew, Buck Hodges). Buck was kind enough to forward me on to Aaron Halberg who gave this reply:

"This is the last bug I fixed for SP1 – passing dynamically generated properties through to solutions/projects. Solving it is rather complicated, unfortunately.

I don’t think that his current workaround will do the trick, for example – he’s expecting the Version property to still be set when CoreCompileSolution is executed, and it won’t be, since he modified it in the global context and it wasn’t passed through to the recursive calls that result in CoreCompileSolution being executed.

He could override the CallCompile target to include the fix I added for SP1:

<Target Name="CallCompile"
DependsOnTargets="$(CoreCompileDependsOn)">

<PropertyGroup>
<OutDir Condition="'%(ConfigurationToBuild.PlatformToBuild)' !=
'Any CPU' ">
$(BinariesRoot)\%(ConfigurationToBuild.PlatformToBuild)\%(ConfigurationToBuild.FlavorToBuild)

</OutDir>

<OutDir Condition=" '%(ConfigurationToBuild.PlatformToBuild)' == 'Any CPU' ">$(BinariesRoot)\%(ConfigurationToBuild.FlavorToBuild)\

</OutDir>
</PropertyGroup>

<MSBuild Projects="$(MSBuildProjectFile)"
Properties="BuildAgentName=$(BuildAgentName); BuildAgentUri=$(BuildAgentUri); BuildDefinitionName=$(BuildDefinitionName); BuildDefinitionUri=$(BuildDefinitionUri); [... Removed for space ] TestResultsRoot=$(TestResultsRoot); $(CustomPropertiesForBuild)"
Targets="CoreCompile">
<output TaskParameter="TargetOutputs" ItemName="CompilationOutputs" />
<msbuild>

<OnError ExecuteTargets="SetBuildBreakProperties;OnBuildBreak;" />

</Target>

Then, after setting the version property in his BuildNumberOverrideTarget, he would do:

Version=$(Version);$(CustomPropertiesForBuild)

The only alternatives to that option that I can think of involve somehow storing the version property so that it can be set again in the appropriate context. For example, he could use the WriteLinesToFile task to write it out to a text file in the BuildNumberOverrideTarget and then do something like:<output taskparameter="Lines" itemname="VersionFileLines">

%(SolutionToBuild.Properties);Version=%(VersionFileLines.Identity)

…or:<output taskparameter="Lines" itemname="VersionFileLines">

Version=$(Version);$(CustomPropertiesForBuild)

The former adds the version property to the SolutionToBuild Properties metadata for each solution. The latter adds it to the CustomPropertiesForBuild property."

I followed the first option but both work well. Choose the one that best fits your style. I haven't seen this information on Aaron's or Buck's blog so I posted it here. If I see it in the near future I will post a link that direction.

Update: Here's Aaron's post: http://blogs.msdn.com/aaronhallberg/archive/2008/05/12/orcas-sp1-tfs-build-changes-part-2.aspx

Playing Know-It-All

I decided to finally update my trial version of VSTS 2008 to the released version (I was going from RC to RTM). What a pain! However, I did have plenty of time to visit some forums and update my blog.

I usually keep to myself and only answer questions when asked, but recently I have been trying to put myself "out there" and see if I can help others who find themselves in the same situation that I have been in. The only draw back is that you really never know how much help you were to the other party (unless of course they pile praises and acclamations galore stating such :). Secondly you have to overcome the inhibition that what you have to say may actually be of some importance to the other party (which can be a major challenge in and of itself).

Well, the upgrade is complete... and actually it was the uninstall of the RC bits that was the pain. The install of the RTM version was rather nice.

TeamBuild 2008

I was going to blog on TFS 2008 and TeamBuild 2008 together, but I figured I had better split it up to give the TeamBuild guys their due props!

We tried to use TeamBuild 2005 without much success as it didn't come with the expected Continuous Integration (not that it advertised it would, but the integration into any existing CI server was out of the question). Yes, we even tried the CI that was published in the MSDN article without success (it ran just fine, but wasn't useful for our resources or build configurations).

I went as far as writing a CruiseControl.Net plugin to try and use the TeamBuild TFS publishing capabilities (see previous post for more information).

When Micorsoft approached us asking us to join the TAP program they took particular interest in our CCNet setup and how TeamBuild 2008 would fare in a head to head comparison. We are now 3 weeks into that comparison and I have to say that TeamBuild is holding its own so far (and in some cases is the clear favorite).

Here is a quick run down of the comparison without trying to draw up charts, graphs, and tables:

• TeamBuild integration into TFS server and Visual Studio 2008 easily beat out the implementation for CCNet. This goes for the times when TFS is down for backup or other issues, you don't have to worry about a quiet period or period when the CI server shouldn't run.
• CCNet setup and ease of use beats TeamBuild, but not by much and it should be qualified with the fact that I have been setting up CruiseControl projects for almost 6 years now. With some documentation and help from the excellent folks at Raleigh I got the builds running as I wanted them to run, not just the default way that was shipped.
• CCNet dashboard and tray apps are more feature rich
• TeamBuild's CI options are more constrictive. You are only allowed a single trigger type and cannot combine them.
• TeamBuild's manual force of a build is more user friendly than CCNet has to offer and you can override properties being passed in (Huge!!).
• CCNet is a lot easier to customize to an organizations process.
• Extensibility would have to be a dead heat for me now. I have been able to create plug-ins for both CCNet and Team Build without much problem.

Not to take anything away from CruiseControl as I am a huge fan of the community and those who work hard to keep it at the head of the Continuous Integration movement, if a company is choosing to use Team Foundation Server as its source control, I would have to recommend they also use TeamBuild 2008.

For everybody else, CruiseControl is the hands down favorite and most usable CI platform on the net.

TFS 2008 Early adoption

I would like to a minute and record some of the effort (or lack thereof) of converting to TFS 2008 and TeamBuild (code named Orcas) from TFS 2005 no TeamBuild (Whidbey).

The guys at Microsoft have done an excellent job with this next release! Normally I am one of their biggest critics as they normally don't do things my way :). However, I have been a fan of the Team Foundation Server source control and project management suite since I was forced to migrate to it (instead of Subversion as I had planned). Understandably TFS 2005 had some growing problems as all new implementations are plagued with. I don't have to mention the installation as the first and foremost issue that most of us that had to administrate the system ran into. We ran into a few other issues, but for the most part TFS ran fine and the issues could be jotted down to our own growing pains.

So I would like to put my voice with the others who are blogging on how different TFS 2008 is than TFS 2005, and say that the setup is awesome! Well, at least a considerable jump from where it was originally. Hopefully the TAP program participants were able to sort out most of the more sinister setup issues, but it works like you would expect. Set the initial settings and click next and then come back a few hours later (depending on data size) and click Finish. Excellent!!!

We have been running 2008 server (with VS 2005 client) for a couple of months now with few interruptions. As an SCM, TFS is sure coming along nicely.

TeamBuild and CC.Net

I just finished coding a plugin that ties TeamBuild into the Continuous Integration (CI) environment of CruiseControl.Net. TeamBuild has been without a satisfactory CI implementation (although, that might change with Orcas), so out of love for continuous integration and a need to track builds in Team Foundation Server (TFS), I went to work.

The initial build was clunky and, with my lack of knowledge of TeamBuild, didn't work very well. The second generation works much better. I have added a logger that will monitor the running build and pump build steps into TFS while the build is running (what you would expect, but man was it hard to get there:). With the new task and along with some editing of the workspace.xml file to help keep the workspace slim and trim, I've been able to convert a few of the myriads of builds to TeamBuild (I am working on the rest in the coming weeks).

One of the problems that I am still having is with the way the Visual Studio view handles hyperlinks to web sites versus file shares. I publish the log as a link to the CruiseControl.Net dashboard, but when activated, it pulls the page into Visual Studio as source (haven't figured out how to get around that one).

A second, which is more important (that's why I listed it second), is that I haven't yet figured out how to update the GlobalList so that the builds are available for reports (still working on it though).

The logger is currently only useful inside the TeamBuild plugin, but with a little tweaking can be made to report any msbuild build to TFS (not sure you want to clutter things like that or not, but one never knows).

I have offered the plugin to Martin Woodward and his TFS plug-in for CruiseControl.Net project, but in the event that it doesn't make it I am more than happy to share (of course with the express permission of my current company).