By Peter Bell

Subversion for Designers? For Clients?

The combination of svn for source control and Ant for managing your builds is hard to beat, but how do you handle builds that need to include templates being edited by designers or files being uploaded by end users of a cms?

One of the great things about subversion is that you can have a centralized repository of resources with the ability to access any version and to handle branching and merging. It's also fairly straightforward to create an automated build process for testing and production servers, but how do you incorporate assets that you don't control into the build?

Dealing with Designers
We build most of our apps for agencies/design firms who often then want to be able to customize the view templates using something like Dreamweaver to tweak the look and feel of the HTML and css to meet their end clients needs. This is great as it means we don't have to deal with small look and feel changes (which can be a real time sink), but we run into issues with overwriting of templates because even though we make heavy use of view helpers to abstract logic, sometimes we have to make changes to the same templates our design partners are working on and occasionally that becomes an issue.

At first, the solution seemed obvious - move the templates to source control. Unfortunately, for people you don't control and who have no background in software, persuading them to use a svn plugin for Dreamweaver and then to fire off an Ant build task to see every tiny change they make in a template is a hard sell. Also, with the number of small changes they make, the number of commits would be an issue. A local development environment is also a non starter as they don't want to mess with web servers locally and often need to show changes in real time to remote end clients.

I've been thinking about setting up a "design server" which would automatically check out a version of the code that they could then work on and I'd run some kind of scheduled task to commit the code every night for which changes had been made or something. That way the designers could continue to use Dreamweaver and FTP to play with the files and I'd hide svn while still getting most of the benefits (they wouldn't be able to roll back to same-day changes, but that hasn't been a request).

Any thought on a seamless way to get HTML designers you don't control or have strong influence over to fit into a source control based workflow?

Coping with Clients
Another issue is finding the best way to handle files uploaded by end clients using a content management system. In some ways it would be very nice to commit such changes to svn as it would allow for svn to provide full snapshots of the entire web application and would mean that automated builds from svn could be used for recovery if a web server failed.

The other approach would be to have a separate solution for backing up client files, but it seems a little strange to have two separate backup/snapshot solutions which would then need to be manually synchronized to successfully restore a server - one for code and another for other files/assets. I suppose again the other solution would be to treat a certain directory on the production server as a checkout from svn and to automatically commit to source control for every day that files had changed.

Any thoughts on the practicalities of automating commits? Any thoughts on mixing up template, code and possibly file commits into a single system? On the one hand it would be nice to have a central repository of everything required to rebuild a site - on the other would the code changes and commits be overwhelmed by all of the noise being generated by the automated file and template commits?

Any thoughts appreciated!

Comments
I have worked with these scenario's in the past, and found a couple solutions that worked for me. First off its great when the designer is willing to work with svn. That solves many problems, especially if you have continuos integration configured to fire and run tests using commit hooks. For the other scneario I found that having a "live" branch in my repository that is a nightly snapshot of the live site has been very helpful in making sure things are working correctly. I use a cron job to rsync the files to the testing location and then run the tests and commit the changes. This is also nice because I can always run a diff of my development code against whatever is on the live site.
# Posted By Simeon Bateman | 2/27/08 10:56 AM
The real challenge for me has been when designers work with files that svn doesn't understand and can't merge. Fla files for example. Anytime two people edit them at the same time it causes headaches. Even if you accidentally save the Fla, you get headaches when you try to update and get merge errors which you have to revert. You want to "lock" the files so that they couldn't be edited on a user's source disk, but I haven't found a way to do that. Subversion's locking system seems to deal only with commits, but by that point its too late.
# Posted By Chris | 2/27/08 12:15 PM
I don't deal with designers anymore (thank goodness!) so can't offer much insight there. :)

In regards to file uploads, etc we discussed this recently having lost some files a client uploaded. We discussed putting these into SVN but we have some applications that have literally thousands of files. After some discussion we decided (for now) that only code related to building the core application should be stored in SVN. Uploaded files, and data stored in the db should be handled independently.

I agree it does sound nice to deploy an entire snapshot of an application at any point in time though... :) Will be interested to hear what others have to say on this...
# Posted By Jim Priest | 2/27/08 12:55 PM
@Chris, Good point. Luckily that isn't an issue for us :->

@Jim, Yeah, on balance we're gonna use svn for code (including templates) and just rsynch or similar to handle the file backups. Not quite as neat, but a little more practical.
# Posted By Peter Bell | 2/27/08 2:00 PM
I'm glad that our css/html guys are quite happy to run local development server and use svn.

Regarding files uploaded using CMS I would consider if it's possible to have database as central storage for files as well as any other data changed using CMS. URLs would have to point to some script that checks if there is cached copy somewhere and if not go to database and get it. As long as amount of data is reasonable this works pretty well and clarifies backups a lot.

One thing we are considering is relatively powerful box running dedicated development servers for each person who needs it using VMware. This would reduce local workstation memory etc. requirements and make it really easy deploy new development servers when needed. Related plan is to use svn for all web server configuration data - most likely this would have to be done with some kind of combination of svn and Ant script that actually configures/reconfigures specific server instance.
# Posted By Tero Pikala | 2/27/08 4:27 PM
I've just started tinkering with this sort of problem and I am looking at Adobes Version Que Server which can deal with all things adobe (including dreamweaver and html I think).

If your designers are using adobe products this could be an option as it hooks directly into adobe file dialogues. I'm not sure about integrating with non adobe products though which may be an issue.
# Posted By Doug Cain | 2/28/08 2:49 AM
Hello, Peter.

If I understood correctly, your build depends on files maintained by your developers and on files maintained by outside designers you have no control over.

And both these files together belong to the same branch right? So this is not about creating branch-specific builds and the designer-maintained files belong to a different branch?

And the problem is that the designers (presumably) can't publish their work directly to svn. Instead they coordinate their own work as they see fit and eventually publish the results via ftp.

And, in order to ensure repeatability, each file consumed by the build script should be versioned in the same branch of the same repository (so each build corresponds exactly to one revision of the repository with no exceptions whatsoever), and that's why your developers need to commit the files from the ftp server to svn and forcing the build script to read files only from svn. Right?

Someone posted his concerns about conflicts. There is no way around it: if the designers maintain unmergeable files, they need to coordinate their work somehow. And assuming they used svn, svn just reveals any problems they might have in the way they coordinate their work. Svn doesn't introduce any new complexity.

Without svn the designers need to fetch the current version from somewhere (ftp get?), tell other designers not to work on it (email?), modify the file, and publish it (ftp put?).

I don't see how svn makes this different or more complicated: just
s/ftp get/svn update/
s/email/svn lock/
s/ftp put/svn commit/

So maybe the designers could learn to live with svn. I admit I have no experience on this, this is just speculation.

I hope I understood the problem correctly and this helps.
# Posted By Ville Oikarinen | 2/28/08 3:44 AM
@Ville, This main issue is build time. The designers are used to editing a file, saving it (2-3 seconds) and viewing it. When you're making lots of tiny Ui changes such as spacing, small changes to font size (should it be 0.8, 0.9, 0.85, 0.88 or 0.87 ems) or little tweaks to colors it's incredibly important that the cycle time from edit to view be kept short. They don't have local dev environments (showing a dev svn is one thing - expecting them to install and maintain a java server on their PC's is a whole other thing), so the entire build process would need to be under 3-5 seconds to be usable. In practice I can't do that. Also, who wants a eight different commits for "made font little bigger", "made font a little smaller" and so on.

The designers are used to FTPing into live server and making tweaks there. That is where the idea comes of giving them a test server that is then automatically committed every evening where there was at least some kind of change. It does leave the issues of what to do if manual merges are required though . . .
# Posted By Peter Bell | 2/28/08 9:18 AM
@Doug,

Interesting! Just had a look at the product:
http://www.adobe.com/products/creativesuite/versio...

Main issue is a little bit of reluctance to add another moving part (and one that would be DW dependent - although most of our design partners do use DW). Also would be a little annoying personally as when I make changes I use CF Eclipse! Would also have to look into stuff like automating builds (It's Java so I'm guessing Ant integration wouldn't be a big deal).

Definitely an interesting approach!
# Posted By Peter Bell | 2/28/08 9:23 AM
Oh, so I did misunderstand the whole scenario :)

Do the designers really need the whole deployed server for making UI tweaks? I have found that with a mock server it's even easier to reproduce certain error flows etc.

If the do need the whole server, then their workspace is actually split between their own workstation and the deployed server. They modify the files locally and view the results remotely, and some synchronization magic happens in between.

The local part could be a svn working copy, and the synchronization between the parts would be easy to automate (ftp, scp, ...), and the script would be fast.

The number of commits wouldn't increase, because the designer can make lots of synchronizations before commiting the svn working copy half of his/her workspace.

Taking things a little further, the target for synchronizing the local working copy to the server could be the same target that does the full deployment, with proper use of the uptodate task of ant and some timestamp generating. (Even with Windows, a "give me all classes and test statuses" run of a workspace of almost 200 projects is a matter of seconds when everything is uptodate.) But maybe this isn't needed.
# Posted By Ville Oikarinen | 3/12/08 10:53 AM
Interesting approach. That culd definitely work. My main concern is that now I need svn client, working copy of Java, ant build scripts and a way for the non-technical designer to fire all of that off. I'm worried I would spend my life troubleshooting the designers machines. I'm really looking for an approach where all the designer needs on their computer is FTP and a text editor of their choice as they all have that. Raising those requirements on the designers computers makes life MUCH more difficult and with a large number of potential resellers I'm not sure I want to invest the time in each ones configuration every time they get a new computer, etc.

Hmmm . . .
# Posted By Peter Bell | 3/12/08 10:56 AM
If you are working with template files you could code a mechanism that creates a design_work zip for each one. That would contain a folder structure with an already generated site page (like the one found in the cache) based on the template, the images folder, the js folder and the css folder. Ideally with relative paths. You can add a BASE tag in the design_work sample html file.
So now they can add images and edit css files. No template mods though. If they edit the sample html file the y ll need to pass it to the devs for them to integrate new layout elements into the real templates.
When they get to a satisfactory result on their local copy they can ftp copy the local images and modded css files to the server. Instant live preview. Server holds a svn working copy. You have a midnight commit that would put their new images and css mods to svn so the devs get them too.
Just a though as I'm stuck in svn land with my designers right now too :)
# Posted By cosmin | 8/20/08 8:11 AM
@Cosmin, Interesting ideas - thanks!
# Posted By Peter Bell | 8/20/08 10:49 AM
BlogCFC was created by Raymond Camden. This blog is running version 5.005.