Archiv der Kategorie: ArchServer

ArchServer Project found on http://www.archserver.org

Howto structure the Git-Repository (of ArchServer.org)

Hello,

a little while ago, I felt the urge to propose a new git structure to my fellows at the ArchServer.org team. I guess the explanation on the why could be helpful for others in the git community as well.

Currently we do have one repository with several packages in one git repository (server-core, server-extra, server-community). This was due to the fact, that the base-distribution (Arch Linux that is) uses SVN as their VCS and we were basically migrating their structure in an easy manner to our own systems.

Right now, we have one branch for each named release (e.g. redgum). Since we are planning to maintain at least two named releases (e.g. redgum and spruce) in parallel, and we have also a testing phase in each release, I suggested two branches for a named release (redgum, redgum-testing). This suggestion was accepted in the mailing list.

I have looked, how other distros with a named release are doing it, as well as for best practices in the git world (well, I really did this for the company I do work for). Fedora is using Git as their repository as well, and git is quite popular, so there is a wealth of information on this topic out there.

PROPOSAL

I propose to restructure the ArchServer.org git repositories to the following structure:

server-core.git/kernel26-lts –> server-core/kernel26-lts.git
server-core.git/aif –> server-core/aif.git

In each of those newly created repositories we will then create at least two branches (e.g. redgum and redgum-testing) for each release. The master-Branch is a special branch, in that it is a development branch. Changes in there will go into release branches, but packages from the master-branches will never be put into a release-version directly. Packages always need to get created from the corresponding release branch, to establish a clean workflow for sign-offs of packages.

To let this work, we will need to adapt our dbscripts slightly, but this should not be too much of a hassel. The git repository could be restructured pretty much automatically and the current already available history could be migrated as well.

Furthermore I suggest to use a tool like repo or fedpkg, adapt it to our needs and implement this into our workflow. Package Updates can then be still done via the usual git commands, but to get a friendlier workflow, this tool can be used.

EXPLANATION

Let me first explain, why do we want to separate each package into its own repository like suggested by git and other distros like e.g. Fedora.

Git suggests to use a single repository for each project/package (a short explantion of this issue can be found on StackOverflow). Basically it comes to the point, that the usage of a repository per package is a best-practice in the git world, and best-pratices do have a sense, like shown in the previous link.

So where is our benefit in splitting repositories (server-core, server-extra and server-community) into several smaller ones (e.g. server-core/kernel26-lts.git)? IMHO the benefit is ease-of-use.

A TU (Trusted User) can then easily get one package, switch to the correct branch (e.g. redgum-testing) make the changes and push these changes back to the main repository. As soon as the package is out of the testing phase, the TU or somebody else can sign-off the changes and merge the package into the release branch (e.g. redgum). We could even say, that only signed packages are allowed to cross the border of the branches, I am unsure, if this could be done by git itself, but it could be a step in the workflow.

So, where is the difference now to the current structure?

Right now, as soon as a TU has made changes and pushed them into the right branch, other TUs will most probably work on other (or even the same) packages, meaning that the TU cannot easily go to the package and merge it into the release branch, he would merge all changes in the whole repository into the branch. The TU has the burden to look up the right commit-ID in the log of the whole repository (remember, git looks on the whole repository and not on sub-folders) and cherry-pick this change (even worse: changes). This does not seem to be right IMHO and it is against the best-practice.

If we are going to split all our repos (server-core, …) into several smaller git repositories, the look and feel of our repository handling will change slightly. To provide a consistent approach for all repositories (e.g. make a new branch for a new release), we will have to provide some tool to make the life of the TUs easier, thats what google is doing with repo and Fedora is doing with fedpkg (see: https://fedorahosted.org/fedora-packager/browser/src/fedpkg.py). These tools offer some functionality not possible with git (at least as far as I know of, see: StackOverflow) like branching of several sub-projects, using the same branch for all sub-projects, committing changes in several sub-projects, etc. Also, there does exsits another alternatives (like git subtree, see: Git Subtree Documentation), these do not fit our needs very well.

Another advantage of this new structure is, that it could be used for automatic builds of packages as well. Right now, pkgbuild.com for example, starts a new build of a package by request. In the software engineering world there is something that is called Continuuous Integration (CI), which builds a package as soon as a change in the repository appears. The tool I am using for this right now (Hudson/Jenkins) does support the build of unix packages as well, so in the long run, it could be worthwhile to investigate this for our use-case as well.

So, WDYT? What are your thoughts on this one? I guess, that this is the way to go, even though it is not for RC4 or even redgum, but soon afterwards, I would suggest is the latest to implement this approach.

Connect from Host to VBox guest via serial console

Today I need to connect to the serial console of a VBox instance to test the serial console boot and connection of the guest system (ArchServer that is). There seem to be a lot of ways on how to do this, I am just explaining, what works for me, using Arch Linux as the Host:

  • socat UNIX-CONNECT:/home/triplem/com1 TCP-LISTEN:8040
  • telnet localhost 8040


This was not working, in that I always received strange Characters in my terminal as soon as I used e.g. the cursor keys. This did not work, even after changing the terminal emulation to VT100 and all others.


The same is true for socat unix-client:/home/triplem/com1 stdout

I do not seem to be able to work with this kind of stuff, the following tip was not working as well:
Work with VBox and serial console.

My ALM thoughts

Yesterday we released a new ALM Stack based on ArchServer, which replaces the beforehand already known „Arch Linux Development Stack„. I guess the name change was needed due to the fact that the underlying architecture changed from Arch Linux to ArchServer and furthermore the ALM in the name shows more appropriate what the purpose of this stack really is.

The goal of this software stack is really, to release a full working stack of tools needed for a Application Lifecycle Management. The selection we chose (Maven, Hudson, Nexus, Sonar and Redmine) is pretty much a best-of-breed solution. One of the main ALM-Software components (IDE) is still missing though. Since this is not a server-side component, it would be really hard to provide this as well. But I guess we should provide an instance of Eclipse with all needed Plugins pre-installed (Mylyn with Redmine Plugin, Git, m2eclipse and probably more…). The same could be done for IntelliJ Idea, which I do like a lot as well, lately.

Something, which should be implemented as well, is most probably ReviewBoard or Gerrit for Reviewing code changes. This is IMHO not needed in all cases of software development, but in some (eg. the Android development and/or the KDE developers).

A real nice thing would be to provide a kind of „Installation procedure“ to install all components in one step with some personalizations as well (e.g. Hostname of the ALM Stack host). Furthermore some real documentation is needed, so that the „customer“ will find the answers to all their questions.

If you would like to participate, do not hesitate to write an email to me 😉

ArchServer – ALM Stack

Please take a closer look onto the newest addition of the ArchServer distribution. I have just published a ALM (Application Lifecycle Management) Stack for this distro with all ALM related tools already installed.

The stack contains the following components:

You can find all scripts to install such a stack (I named the stack Archlinux Development Stack earlier on, but it should now be called ArchServer ALM Stack), on Github. There is also additional information about this stack. In the near future I hope to add more components as well as more documentation. The documentation should then be also put into the ArchServer.org wiki.

The Virtualbox image itself can be found on our mirror at http://mirror.archserver.org/iso/vbox/ArchServer-ALMStack-20110101-i686.vdi.lzma. The image is lzma-compressed, so you need lzma to decompress the image.

ArchServer Kernel Update

The last weekend was a busy weekend for me. I have updated the kernel of ArchServer to its latest incarnation 2.6.32.11. If you find time, please try out ArchServer and test the newest Kernel version.