summaryrefslogtreecommitdiff
path: root/git-debrebase.5.pod
blob: 52fb60b952d331c7632ab3abc2c8b424f80de624 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
=head1 NAME

git-debrebase - git data model for Debian packaging

=head1 INTRODUCTION

git-debrebase is a tool for representing in git,
and manpulating,
Debian packages based on upstream source code.

The Debian packaging
has a fast forwarding history.
The delta queue (changes to upstream files) is represented
as a series of individual git commits,
which can worked on with rebase,
and also shared.

=head2 DISCUSSION

git-debrebase is designed to work well with dgit.
git-debrebase can also be used in workflows without source packages,
for example to work on Debian-format packages outside or alongside Debian.

git-debrebase
itself is not very suitable for use by Debian derivatives,
to work on packages inherited from Debian,
because it assumes that you want to throw away any packaging
provided by your upstream.
However, use of git-debrebase in Debian does not make anything harder for
derivatives, and it can make some things easier.

When using gitk on branches managed by git-debrebase,
B<gitk --date-order>, B<gitk --first-parent>
and B<gitk -- :.> (or B<gitk .>)
produce more useful output than the default.

=head1 TERMINOLOGY

=over

=item Pseudomerge

A merge which does not actually merge the trees;
instead, it is constructed by taking the tree
from one of the parents
(ignoring the contents of the other parents).
These are used to make a rewritten history fast forward
from a previous tip,
so that it can be pushed and pulled normally.
Manual construction of pseudomerges can be done with
C<git merge -s ours>
but is not normally needed when using git-debrebase.

=item Packaging files

Files in the source tree within B<debian/>,
excluding anything in B<debian/patches/>.

=item Upstream

The version of the package without Debian's packaging.
Typically provided by the actual upstream project,
and sometimes tracked by Debian contributors in a branch C<upstream>.

Upstream contains upstream files,
but some upstreams also contain packaging files in B<debian/>.
Any such non-upstream files found in upstream
are thrown away by git-debrebase
each time a new upstream version is incorporated.

=item Upstream files

Files in the source tree outside B<debian/>.
These may include unmodified source from upstream,
but also files which have been modified or created for Debian.

=item Delta queue

Debian's changes to upstream files:
a series of git commits.

=item Quilt patches

Files in B<debian/patches/> generated for the benefit of
dpkg-source's 3.0 (quilt) .dsc source package format.
Not used, often deleted, and regenerated when needed
(such as when uploading to Debian),
by git-debrebase.

=item Interchange branch; breakwater; stitched; laundered

See L</BRANCHES AND BRANCH STATES - OVERVIEW>.

=item Anchor; Packaging

See L</BRANCH CONTENTS - DETAILED SPECIFICATION>.

=item ffq-prev; debrebase-last

See L</STITCHING, PSEUDO-MERGES, FFQ RECORD>.

=back

=head1 DIAGRAM

           ------/--A!----/--B3!--%--/--> interchange view
                /        /          /      with debian/ directory
               %        %          %       entire delta queue applied
              /        /          /        3.0 (quilt) has debian/patches
             /        /          3*       "master" on Debian git servers
            /        /          /
           2*       2*         2
          /        /          /
         1        1          1    breakwater branch, merging baseline
        /        /          /     unmodified upstream code
    ---@-----@--A----@--B--C      plus debian/ (but no debian/patches)
      /     /       /                     no ref refers to this: we
   --#-----#-------#-----> upstream        reconstruct its identity by
                                           inspecting interchange branch
    Key:

      1,2,3   commits touching upstream files only
      A,B,C   commits touching debian/ only
      B3      mixed commit (eg made by an NMUer)
      #       upstream releases

     -@-      anchor merge, takes contents of debian/ from the
     /         previous `breakwater' commit and rest from upstream

     -/-      pseudomerge; contents are identical to
     /         parent lower on diagram.

      %       dgit- or git-debrebase- generated commit of debian/patches.
              `3.0 (quilt)' only; generally dropped by git-debrebase.

      *       Maintainer's HEAD was here while they were editing,
              before they said they were done, at which point their
              tools made -/- (and maybe %) to convert to
              the fast-forwarding interchange branch.

      !       NMUer's HEAD was here when they said `dgit push'.
              Rebase branch launderer turns each ! into an
              equivalent *.

=head1 BRANCHES AND BRANCH STATES - OVERVIEW

git-debrebase has one primary branch,
the B<interchange branch>.
This branch is found on Debian contributor's workstations
(typically, a maintainer would call it B<master>),
in the Debian dgit git server as the suite branch (B<dgit/dgit/sid>)
and on other git servers which support Debian work
(eg B<master> on salsa).

The interchange branch is fast-forwarding
(by virtue of pseudomerges, where necessary).

It is possible to have multiple different interchange branches
for the same package,
stored as different local and remote git branches.
However, divergence should be avoided where possible -
see L</OTHER MERGES>.

A suitable interchange branch can be used directly with dgit.
In this case each dgit archive suite branch is a separate
interchange branch.

Within the ancestry of the interchange branch,
there is another important, implicit branch, the
B<breakwater>.
The breakwater contains unmodified upstream source,
but with Debian's packaging superimposed
(replacing any C<debian/> directory that may be in
the upstream commits).
The breakwater does not contain any representation of
the delta queue (not even debian/patches).
The part of the breakwater processed by git-debrebase
is the part since the most reecent B<anchor>,
which is usually a special merge generated by git-debrebase.

When working, locally,
the user's branch can be in a rebasing state,
known as B<unstitched>.
While a branch is unstitched,
it is not in interchange format.
The previous interchange branch tip
is recorded,
so that the previous history
and the user's work
can later be
stitched into the fast-forwarding interchange form.

An unstitched branch may be in
B<laundered>
state,
which means it has a more particular special form
convenient for manipulating the delta queue.

=head1 BRANCH CONTENTS - DETAILED SPECIFICATION

It is most convenient to describe the
B<breakwater>
branch first.
A breakwater is B<fast-forwarding>,
but is not usually named by a ref.
It contains B<in this order> (ancestors first):

=over

=item Anchor

An B<anchor> commit,
which is usually a special two-parent merge:

The first parent
contains the most recent version, at that point,
of the Debian packaging (in debian/);
it also often contains upstream files,
but they are to be ignored.
Often the first parent is a previous breakwater tip.

The second parent
is an upstream source commit.
It may sometimes contain a debian/ subdirectory,
but if so that is to be ignored.
The second parent's upstream files
are identical to the anchor's.
Anchor merges always contain
C<[git-debrebase anchor: ...]>
as a line in the commit message.

Alternatively,
an anchor may be a single-parent commit which introduces
the C<debian/> directory and makes no other changes:
ie, the start of Debian packaging.

=item Packaging

Zero or more single-parent commits
containing only packaging changes.
(And no quilt patch changes.)

=back

The
B<laundered>
branch state is B<rebasing>.
A laundered branch is based on a breakwater
but also contains, additionally,
B<after> the breakwater,
a representation of the delta queue:

=over

=item Delta queue commits

Zero or more single-parent commits
contaioning only changes to upstream files.

=back

The merely
B<unstitched>
(ie, unstitched but unlaundered)
branch state is also B<rebasing>.
It has the same contents as the laundered state,
except that it may contain,
additionally,
in B<in any order but after the breakwater>:

=over

=item Linear commits to the source

Further commit(s) containing changes to
to upstream files
and/or
to packaging,
possibly mixed within a single commit.
(But not quilt patch changes.)

=item Quilt patch addition for `3.0 (quilt)'

Commit(s) which add patches to B<debian/patches/>,
and add those patches to the end of B<series>.

These are only necessary when working with
packages in C<.dsc 3.0 (quilt)> format.
For git-debrebase they are purely an output;
they are deleted when branches are laundered.
git-debrebase takes care to make a proper patch
series out of the delta queue,
so that any resulting source packages are nice.

=back

Finally, an
B<interchange>
branch is B<fast forwarding>.
It has the same contents as an
unlaundered branch state,
but may (and usually will) additionally contain
(in some order,
possibly intermixed with the extra commits
which may be found on an unstitched unlaundered branch):

=over

=item Pseudomerge to make fast forward

A pseudomerge making the branch fast forward from
previous history.
The contributing parent is itself in interchange format.
Normally the overwritten parent is
a previous tip of an interchange branch,
but this is not necessary as the overwritten
parent is not examined.

If the two parents have identical trees,
the one with the later commit date
(or, if the commit dates are the same,
the first parent)
is treated as
the contributing parent.

=item dgit dsc import pseudomerge

Debian .dsc source package import(s)
made by dgit
(during dgit fetch of a package most recently 
uploaded to Debian without dgit,
or during dgit import-dsc).

git-debrebase requires that
each such import is in the fast-forwarding
format produced by dgit:
a two-parent pseudomerge,
whose contributing parent is in the
non-fast-forwarding
dgit dsc import format (not described further here),
and whose overwritten parent is
the previous interchange tip
(eg, the previous tip of the dgit suite branch).

=back

=head1 STITCHING, PSEUDO-MERGES, FFQ RECORD

Whenever the branch C<refs/B> is unstitched,
the previous head is recorded in the git ref C<refs/ffq-prev/B>.

Unstiched branches are not fast forward from the published
interchange branches [1].
So before a branch can be pushed,
the right pseudomerge must be reestablished.
This is the stitch operation,
which consumes the ffq-prev ref.

When the user has an unstitched branch,
they may rewrite it freely,
from the breakwater tip onwards.
Such a git rebase is the default operation for git-debrebase.
Rebases should not go back before the breakwater tip,
and certainly not before the most recent anchor.

Unstitched branches must not be pushed to interchange branch refs
(by the use of C<git push -f> or equivalent).
It is OK to share an unstitched branch
in similar circumstances and with similar warnings
to sharing any other rebasing git branch.

[1] Strictly, for a package
which has never had a Debian delta queue,
the interchange and breakwater branches may be identical,
in which case the unstitched branch is fast forward
from the interchange branch and no pseudomerge is needed.

When ffq-prev is not present,
C<refs/debrebase-last/B> records some ancestor of refs/B,
(usually, the result of last stitch).
This can be used to quickly determine whether refs/B
is being maintained in git-debrebase form.

=head1 OTHER MERGES

Note that the representation described here does not permit
general merges on any of the relevant branches.
For this reason the tools will try to help the user
avoid divergence of the interchange branch.

See dgit-maint-rebase(7)
for a discussion of what kinds of behaviours
should be be avoided
because
they might generate such merges.

Automatic resolution of divergent interchange branches
(or laundering of merges on the interchange branch)
is thought to be possible,
but there is no tooling for this yet:

Nonlinear (merging) history in the interchange branch is awkward
because it (obviously) does not preserve
the linearity of the delta queue.
Easy merging of divergent delta queues is a research problem.

Nonlinear (merging) history in the breakwater branch is
in principle tolerable,
but each of the parents would have to be, in turn,
a breakwater,
and difficult questions arise if they don't have the same anchor.

We use the commit message annotation to
distinguish the special anchor merges from other general merges,
so we can at least detect unsupported merges.

=head1 LEGAL OPERATIONS

The following basic operations follows from this model
(refer to the diagram above):

=over

=item Append linear commits

No matter the branch state,
it is always fine to simply git commit
(or cherry-pick etc.)
commits containing upstream file changes, packaging changes,
or both.

(This may make the branch unlaundered.)

=item Launder branch

Record the previous head in ffq-prev,
if we were stitched before
(and delete debrebase-last).

Reorganise the current branch so that the packaging
changes come first,
followed by the delta queue,
turning C<-@-A-1-2-B3> into C<...@-A-B-1-2-3>.

Drop pseudomerges and any quilt patch additions.

=item Interactive rebase

With a laundered branch,
one can do an interactive git rebase of the delta queue.

=item New upstream rebase

Start rebasing onto a new upstream version,
turning C<...#..@-A-B-1-2-3> into C<(...#..@-A-B-, ...#'-)@'-1-2>.

This has to be a wrapper around git-rebase,
which prepares @' and then tries to rebase 1 2 onto @'.
If the user asks for an interactive rebase,
@' doesn't appear in the commit list, since
@' is the newbase of the rebase (see git-rebase(1)).

Note that the construction of @' cannot fail
because @' simply copies debian/ from B and and everything else from #'.
(Rebasing A and B is undesirable.
We want the debian/ files to be non-rebasing
so that git log shows the packaging history.)

=item Stitch 

Make a pseudomerge,
whose contributing parent to is the unstitched branch
and
whose overwritten parent is ffq-prev,
consuming ffq-prev in the process
(and writing debrebase-last instead).
Ideally the contributing parent would be a laundered branch,
or perhaps a laundered branch with a quilt patch addition commit.

=item Commit quilt patches

To generate a tree which can be represented as a
3.0 (quilt) .dsc source packages,
the delta queue must be reified inside the git tree
in B<debian/patches/>.
These patch files can be stripped out and/or regenerated as needed.

=back

=head1 COMMIT MESSAGE ANNOTATIONS

git-debrebase makes annotations
in the messages of commits it generates.

The general form is

  [git-debrebase[ COMMIT-TYPE [ ARGS...]]: PROSE, MORE PROSE]

git-debrebase treats anything after the colon as a comment,
paying no attention to PROSE.

The full set of annotations is:
  [git-debrebase: split mixed commit, debian part]
  [git-debrebase: split mixed commit, upstream-part]
  [git-debrebase: convert dgit import, debian changes]
  [git-debrebase anchor: convert dgit import, upstream changes]

  [git-debrebase upstream-combine . PIECE[ PIECE...]: new upstream]
  [git-debrebase anchor: new upstream NEW-UPSTREAM-VERSION, merge]
  [git-debrebase: new upstream NEW-UPSTREAM-VERSION, changelog]
  [git-debrebase: export and commit patches]

  [git-debrebase convert-from-gbp: drop patches]
  [git-debrebase anchor: declare upstream]
  [git-debrebase pseudomerge: stitch]

  [git-debrebase convert-to-gbp: commit patches]

Only anchor merges have the C<[git-debrebase anchor: ...]> tag.
Single-parent anchors are not generated by git-debrebase,
and when made manually should not
contain any C<[git-debrebase ...]> annotation.

The C<split mixed commit> and C<convert dgit import>
tags are added to the pre-existing commit message,
when git-debrebase rewrites the commit.

=head1 APPENDIX - DGIT IMPORT HANDLING

The dgit .dsc import format is not documented or specified
(so some of the following terms are not defined anywhere).
The dgit import format it is defined by the implementation in dgit,
of which git-debrebase has special knowledge.

Consider a non-dgit NMU followed by a dgit NMU:

            interchange --/--B3!--%--//----D*-->
                         /          /
                        %          4
                       /          3
                      /          2
                     /          1
                    2          &_
                   /          /| \
                  1          0 00 =XBC%
                 /
                /
          --@--A     breakwater
           /
        --#--------> upstream


 Supplementary key:

    =XBC%     dgit tarball import of .debian.tar.gz containing
               Debian packaging including changes B C and quilt patches
    0         dgit tarball import of upstream tarball
    00        dgit tarball import of supplementary upstream piece
    &_        dgit import nearly-breakwater-anchor
    //        dgit fetch / import-dsc pseudomerge to make fast forward

    &'        git-debrebase converted import (upstream files only)
    C'        git-debrebase converted packaging change import

    * **      before and after HEAD

We want to transform this into:

=over

=item I. No new upstream version

 (0 + 00 eq #)
                        --/--B3!--%--//-----D*-------------/-->
                         /          /                     /
                        %          4                     4**
                       /          3                     3
                      /          2                     2
                     /          1                     1
                    2          &_                    /
                   /          /| \                  /
                  1          0 00 =XBC%            /
                 /                                /
                /                                /
          --@--A-----B---------------------C'---D
           /
        --#----------------------------------------->

=item II. New upstream

 (0 + 00 neq #)

                        --/--B3!--%--//-----D*-------------/-->
                         /          /                     /
                        %          4                     4**
                       /          3                     3
                      /          2                     2
                     /          1                     1
                    2          &_                    /
                   /          /| \                  /
                  1          0 00 =XBC%            /
                 /                                /
                /                                /
          --@--A-----B-----------------@---C'---D
           /                          /
        --#--------------------- - - / - - --------->
                                    /
                                   &'
                                  /|
                                 0 00

=back

=head1 SEE ALSO

git-debrebase(1),
dgit-maint-rebase(7),
dgit(1)