diff options
Diffstat (limited to 'dgit.7')
| -rw-r--r-- | dgit.7 | 425 |
1 files changed, 425 insertions, 0 deletions
@@ -0,0 +1,425 @@ +.TH dgit 7 "" "Debian Project" "dgit" +.SH NAME +dgit \- principles of operation +.SH SUMMARY +.B dgit +treats the Debian archive as a version control system, and +bidirectionally gateways between the archive and git. The git view of +the package can contain the usual upstream git history, and will be +augmented by commits representing uploads done by other developers not +using dgit. This git history is stored in a canonical location known +as +.B dgit-repos +which lives on a dedicated git server. + +git branches suitable for use with dgit +can be edited directly in git, +and used directly for building binary packages. +They can be shared using all conventional means for sharing git +branches. +It is not necessary to use dgit to work with dgitish git branches. +However, dgit is (usually) needed in order to convert to or from +Debian-format source packages. +.SH SEE ALSO +.TP +\fBdgit\fP(1) +Reference manual and documentation catalogue. +.TP +\fBdgit-*\fB(7) +Tutorials and workflow guides. See dgit(1) for a list. +.SH MODEL +You may use any suitable git workflow with dgit, provided you +satisfy dgit's requirements: + +dgit maintains a pseudo-remote called +.BR dgit , +with one branch per suite. This remote cannot be used with +plain git. + +The +.B dgit-repos +repository for each package contains one ref per suite named +\fBrefs/dgit/\fR\fIsuite\fR. These should be pushed to only by +dgit. They are fast forwarding. Each push on this branch +corresponds to an upload (or attempted upload). + +However, it is perfectly fine to have other branches in dgit-repos; +normally the dgit-repos repo for the package will be accessible via +the remote name `origin'. + +dgit push will also make signed tags called +.BI archive/debian/ version +(with version encoded a la DEP-14) +and push them to dgit-repos. These are used at the +server to authenticate pushes. + +Uploads made by dgit contain an additional field +.B Dgit +in the source package .dsc. (This is added by dgit push.) +This specifies: a commit (an ancestor of the dgit/suite +branch) whose tree is identical to the unpacked source upload; +the distro to which the upload was made; +a tag name which can be used to fetch the git commits; +and +a url to use as a hint for the dgit git server for that distro. + +Uploads not made by dgit are represented in git by commits which are +synthesised by dgit. The tree of each such commit corresponds to the +unpacked source; there is a +commit with the contents, +and a +pseudo-merge from last known upload - that is, from the contents of +the dgit/suite branch. +Depending on the source package format, +the contents commit may have a more complex structure, +but ultimately it will be a convergence of stubby branches +from origin commits representing the components of the source package. + +dgit expects trees that it works with to have a +.B dgit +(pseudo) remote. This refers to the dgit-created git view of +the corresponding archive. + +The dgit archive tracking view is synthesised locally, +on demand, +by each copy of dgit. +The tracking view is always a descendant of the +dgit-repos suite branch (if one exists), +but may be ahead of it if uploads have been done without dgit. +The archive tracking view is always fast forwarding within +each suite. + +dgit push can operate on any commit which is a descendant of +the suite tracking branch. + +dgit does not make a systematic record of +its imports of orig tarball(s). +So it does not work by finding git tags or branches +referring to orig tarball(s). +The +orig tarballs are downloaded (by dgit clone) into the parent +directory, as with a traditional (non-gitish) dpkg-source workflow. +You need to retain these tarballs in the parent directory for dgit +build and dgit push. +(They are not needed for purely-git-based workflows.) + +dgit repositories could be cloned with standard (git) methods. +However, +the dgit repositories do not contain uploads not made with dgit. +And +for sourceful builds / uploads the orig +tarball(s) will need to be present in the parent directory. + +To a user looking at the archive, changes pushed +in a simple NMU +using dgit look like +reasonable +changes made in an NMU: in a `3.0 (quilt)' package the delta from the +previous upload is recorded in new patch(es) constructed by dpkg-source. +.SH COMBINED SUITES +dgit can synthesize a combined view of several underlying suites. +This is requested by specifying, for +.I suite, +a comma-separated list: +.IP +.IR mainsuite \fB,\fR subsuite ... +.LP +This facility is available with dgit clone, fetch and pull, only. + +dgit will fetch the same package from each specified underlying suite, +separately (as if with dgit fetch). +dgit will then generate a pseudomerge commit +on the tracking branch +.BI remotes/dgit/dgit/ suite +which has the tip of each of the underlying suites +as an ancestor, +and which contains the same as the suite which +has the highest version of the package. + +The package must exist in mainsuite, +but need not exist in the subsuites. + +If a specified subsuite starts with +.B - +then mainsuite is prepended. + +So, for example, +.B stable,-security +means to look for the package in stable, and stable-security, +taking whichever is newer. +If stable is currently jessie, +dgit clone would leave you on the branch +.BR dgit/jessie,-security . + +Combined suites are not supported by the dgit build operations. +This is because those options are intended for building for +uploading source packages, +and look in the changelog to find the relevant suite. +It does not make sense to name a dgit-synthesised combined suite +in a changelog, +or to try to upload to it. + +When using this facility, it is important to always specify the +same suites in the same order: +dgit will not be make a coherent fast-forwarding history +view otherwise. + +The history generated by this feature is not normally suitable +for merging back into upstreams, +as it necessarily contains unattractive pseudomerges. +.SH LIMITATIONS +Because the synthesis +of the suite tracking branches +is done locally based only on the current archive state, +it will not necessarily see every upload +not done with dgit. +Also, different versions of dgit +(or the software it calls) +might import the same .dscs differently +(although we try to minimise this). +As a consequence, the dgit tracking views of the same +suite, made by different instances of dgit, may vary. +They will have the same contents, but may have different history. + +There is no uniform linkage between the tracking branches for +different suites. +The Debian infrastructure +does not do any automatic import of uploads made without dgit. +It would be possible for a distro's infrastructure to do this; +in that case, +different dgit client instances +would see exactly the same history. + +There has been no bulk import of historical uploads into +Debian's dgit infrastructure. +To do this it would be necessary to decide whether to +import existing vcs history +(which might not be faithful to dgit's invariants) +or previous non-Dgit uploads +(which would not provide a very rich history). +.SH READ-ONLY DISTROS +Distros which do not maintain a set of dgit history git repositories +can still be used in a read-only mode with dgit. Currently Ubuntu +is configured this way. +.SH GITATTRIBUTES +git has features which can automatically transform files +as they are being copied between the working tree +and the git history. +The attributes can be specified in the source tree itself, +in +.BR .gitattributes . +See \fBgitattributes\fP(5). + +These transformations are context-sensitive +and not, in general, reversible, +so dgit operates on the principle that +the dgit git history contains the actual contents of the package. +(When dgit is manipulating a .dsc, +it does so in a private area, +where the transforming gitattributes are defused, +to achieve this.) + +If transforming gitattributes are used, +they can cause trouble, +because the working tree files can differ from +the git revision history +(and therefore from the source packages). +dgit warns if it finds a .gitattributes file +(in a package being fetched or imported), +unless the transforming gitattributes have been defused. + +dgit clone +and dgit setup-new-tree +disable transforming gitattributes +by default, +by creating a suitable .git/info/attributes. +See +.B dgit setup-new-tree +and +.B dgit setup-gitattributes +in dgit(1). +.SH PACKAGE SOURCE FORMATS +If you are not the maintainer, you do not need to worry about the +source format of the package. You can just make changes as you like +in git. If the package is a `3.0 (quilt)' package, the patch stack +will usually not be represented in the git history. +.SH FORMAT 3.0 (QUILT) +For a format `3.0 (quilt)' source package, dgit may have to make a +commit on your current branch to contain metadata used by quilt and +dpkg-source. + +This is because `3.0 (quilt)' source format represents the patch stack +as files in debian/patches/ actually inside the source tree. This +means that, taking the whole tree (as seen by git or ls) (i) +dpkg-source cannot represent certain trees, and (ii) packing up a tree +in `3.0 (quilt)' and then unpacking it does not always yield the same +tree. + +dgit will automatically work around this for you when building and +pushing. The only thing you need to know is that dgit build, sbuild, +etc., may make new commits on your HEAD. If you're not a quilt user +this commit won't contain any changes to files you care about. + +You can explicitly request that dgit do just this fixup, by running +dgit quilt-fixup. + +If you are a quilt user you need to know that dgit's git trees are +`patches applied packaging branches' and do not contain the .pc +directory (which is used by quilt to record which patches are +applied). If you want to manipulate the patch stack you probably want +to be looking at tools like git-dpm. +.SH SPLIT VIEW QUILT MODE +When working with git branches intended +for use with the `3.0 (quilt)' source format +dgit can automatically convert a suitable +maintainer-provided git branch +(in one of a variety of formats) +into a dgit branch. + +When a split view mode is engaged +dgit build commands and +dgit push +will, on each invocation, +convert the user's HEAD into the dgit view, +so that it can be built and/or uploaded. + +dgit push in split view mode will push the dgit view to the dgit +git server. +The dgit view is always a descendant of the maintainer view. +dgit push will also make a maintainer view tag +according to DEP-14 +and push that to the dgit git server. + +Split view mode must be enabled explicitly +(by the use of the applicable command line options, +subcommands, or configuration). +This is because it is not possible to reliably tell +(for example) +whether a git tree for a dpkg-source `3.0 (quilt)' package +is a patches-applied or patches-unapplied tree. + +Split view conversions are cached in the ref +dgit-intern/quilt-cache. +This should not be manipulated directly. +.SH FILES IN THE ORIG TARBALL BUT NOT IN GIT - AUTOTOOLS ETC. +This section is mainly of interest to maintainers who want to use dgit +with their existing git history for the Debian package. + +Some developers like to have an extra-clean git tree which lacks files +which are normally found in source tarballs and therefore in Debian +source packages. For example, it is conventional to ship ./configure +in the source tarball, but some people prefer not to have it present +in the git view of their project. + +dgit requires that the source package unpacks to exactly the same +files as are in the git commit on which dgit push operates. So if you +just try to dgit push directly from one of these extra-clean git +branches, it will fail. + +As the maintainer you therefore have the following options: +.TP +\(bu +Delete the files from your git branches, +and your Debian source packages, +and carry the deletion as a delta from upstream. +(With `3.0 (quilt)' this means represeting the deletions as patches. +You may need to pass --include-removal to dpkg-source --commit, +or pass corresponding options to other tools.) +This can make the Debian +source package less useful for people without Debian build +infrastructure. +.TP +\(bu +Persuade upstream that the source code in their git history and the +source they ship as tarballs should be identical. Of course simply +removing the files from the tarball may make the tarball hard for +people to use. +.IP +One answer is to commit the (maybe autogenerated) +files, perhaps with some simple automation to deal with conflicts and +spurious changes. This has the advantage that someone who clones +the git repository finds the program just as easy to build as someone +who uses the tarball. +.LP +Of course it may also be that the differences are due to build system +bugs, which cause unintended files to end up in the source package. +dgit will notice this and complain. You may have to fix these bugs +before you can unify your existing git history with dgit's. +.LP +.SH FILES IN THE SOURCE PACKAGE BUT NOT IN GIT - DOCS, BINARIES ETC. +Some upstream tarballs contain build artifacts which upstream expects +some users not to want to rebuild (or indeed to find hard to rebuild), +but which in Debian we always rebuild. +.LP +Examples sometimes include crossbuild firmware binaries and +documentation. +To avoid problems when building updated source +packages +(in particular, to avoid trying to represent as changes in +the source package uninteresting or perhaps unrepresentable changes +to such files) +many maintainers arrange for the package clean target +to delete these files. +.LP +dpkg-source does not +(with any of the commonly used source formats) +represent deletion of binaries (outside debian/) present in upstream. +Thus deleting such files in a dpkg-source working tree does not +actually result in them being deleted from the source package. +Thus +deleting the files in rules clean sweeps this problem under the rug. +.LP +However, git does always properly record file deletion. +Since dgit's +principle is that the dgit git tree is the same of dpkg-source -x, +that means that a dgit-compatible git tree always contains these +files. +.LP +For the non-maintainer, +this can be observed in the following suboptimal occurrences: +.TP +\(bu +The package clean target often deletes these files, making the git +tree dirty trying to build the source package, etc. +This can be fixed +by using +.BR "dgit -wg" " aka " "--clean=git" , +so that the package clean target is never run. +.TP +\(bu +The package build modifies these files, so that builds make the git +tree dirty. +This can be worked around by using `git reset --hard' +after each build +(or at least before each commit or push). +.LP +From the maintainer's point of view, +the main consequence is that to make a dgit-compatible git branch +it is necessary to commit these files to git. +The maintainer has a few additional options for mitigation: +for example, +it may be possible for the rules file to arrange to do the +build in a temporary area, which avoids updating the troublesome +files; +they can then be left in the git tree without seeing trouble. +.SH PROBLEMS WITH PACKAGE CLEAN TARGETS ETC. +A related problem is other unexpected behaviour by a package's +.B clean +target. +If a package's rules +modify files which are distributed in the package, +or simply forget to remove certain files, +dgit will complain that the tree is dirty. +.LP +Again, the solution is to use +.BR "dgit -wg" " aka " "--clean=git" , +which instructs dgit to use git clean instead of the package's +build target, +along with perhaps +.B git reset --hard +before each build. +.LP +This is 100% reliable, but has the downside +that if you forget to git add or to commit, and then use +.BR "dgit -wg" " or " "git reset --hard" , +your changes may be lost. |