1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
|
Metadata-Version: 1.1
Name: priority
Version: 1.3.0
Summary: A pure-Python implementation of the HTTP/2 priority tree
Home-page: http://python-hyper.org/priority/
Author: Cory Benfield
Author-email: cory@lukasa.co.uk
License: MIT License
Description: Priority: A HTTP/2 Priority Implementation
==========================================
Priority is a pure-Python implementation of the priority logic for HTTP/2, set
out in `RFC 7540 Section 5.3 (Stream Priority)`_. This logic allows for clients
to express a preference for how the server allocates its (limited) resources to
the many outstanding HTTP requests that may be running over a single HTTP/2
connection.
Specifically, this Python implementation uses a variant of the implementation
used in the excellent `H2O`_ project. This original implementation is also the
inspiration for `nghttp2's`_ priority implementation, and generally produces a
very clean and even priority stream. The only notable changes from H2O's
implementation are small modifications to allow the priority implementation to
work cleanly as a separate implementation, rather than being embedded in a
HTTP/2 stack directly.
While priority information in HTTP/2 is only a suggestion, rather than an
enforceable constraint, where possible servers should respect the priority
requests of their clients.
Using Priority
--------------
Priority has a simple API. Streams are inserted into the tree: when they are
inserted, they may optionally have a weight, depend on another stream, or
become an exclusive dependent of another stream.
.. code-block:: python
>>> p = priority.PriorityTree()
>>> p.insert_stream(stream_id=1)
>>> p.insert_stream(stream_id=3)
>>> p.insert_stream(stream_id=5, depends_on=1)
>>> p.insert_stream(stream_id=7, weight=32)
>>> p.insert_stream(stream_id=9, depends_on=7, weight=8)
>>> p.insert_stream(stream_id=11, depends_on=7, exclusive=True)
Once streams are inserted, the stream priorities can be requested. This allows
the server to make decisions about how to allocate resources.
Iterating The Tree
~~~~~~~~~~~~~~~~~~
The tree in this algorithm acts as a gate. Its goal is to allow one stream
"through" at a time, in such a manner that all the active streams are served as
evenly as possible in proportion to their weights.
This is handled in Priority by iterating over the tree. The tree itself is an
iterator, and each time it is advanced it will yield a stream ID. This is the
ID of the stream that should next send data.
This looks like this:
.. code-block:: python
>>> for stream_id in p:
... send_data(stream_id)
If each stream only sends when it is 'ungated' by this mechanism, the server
will automatically be emitting stream data in conformance to RFC 7540.
Updating The Tree
~~~~~~~~~~~~~~~~~
If for any reason a stream is unable to proceed (for example, it is blocked on
HTTP/2 flow control, or it is waiting for more data from another service), that
stream is *blocked*. The ``PriorityTree`` should be informed that the stream is
blocked so that other dependent streams get a chance to proceed. This can be
done by calling the ``block`` method of the tree with the stream ID that is
currently unable to proceed. This will automatically update the tree, and it
will adjust on the fly to correctly allow any streams that were dependent on
the blocked one to progress.
For example:
.. code-block:: python
>>> for stream_id in p:
... send_data(stream_id)
... if blocked(stream_id):
... p.block(stream_id)
When a stream goes from being blocked to being unblocked, call the ``unblock``
method to place it back into the sequence. Both the ``block`` and ``unblock``
methods are idempotent and safe to call repeatedly.
Additionally, the priority of a stream may change. When it does, the
``reprioritize`` method can be used to update the tree in the wake of that
change. ``reprioritize`` has the same signature as ``insert_stream``, but
applies only to streams already in the tree.
Removing Streams
~~~~~~~~~~~~~~~~
A stream can be entirely removed from the tree by calling ``remove_stream``.
Note that this is not idempotent. Further, calling ``remove_stream`` and then
re-adding it *may* cause a substantial change in the shape of the priority
tree, and *will* cause the iteration order to change.
License
-------
Priority is made available under the MIT License. For more details, see the
LICENSE file in the repository.
Authors
-------
Priority is maintained by Cory Benfield, with contributions from others. For
more details about the contributors, please see CONTRIBUTORS.rst in the
repository.
.. _RFC 7540 Section 5.3 (Stream Priority): https://tools.ietf.org/html/rfc7540#section-5.3
.. _nghttp2's: https://nghttp2.org/blog/2015/11/11/stream-scheduling-utilizing-http2-priority/
.. _H2O: https://h2o.examp1e.net/
Changelog
=========
1.3.0 (2017-01-27)
------------------
**API Changes**
- Throw ``PriorityLoop`` when inserting or reprioritising a stream that
depends on itself.
- Throw ``BadWeightError`` when creating or reprioritising a stream with a
weight that is not an integer between 1 and 256, inclusive.
- Throw ``PseudoStreamError`` when trying to reprioritise, remove, block or
unblock stream 0.
- Add a new ``PriorityError`` parent class for the exceptions that can be
thrown by priority.
1.2.2 (2016-11-11)
------------------
**Bugfixes**
- Allow ``insert_stream`` to be called with ``exclusive=True`` but no explicit
``depends_on`` value.
1.2.1 (2016-10-26)
------------------
**Bugfixes**
- Allow insertion of streams that have parents in the idle or closed states.
This would previously raise a KeyError.
1.2.0 (2016-08-04)
------------------
**Security Fixes**
- CVE-2016-6580: All versions of this library prior to 1.2.0 are vulnerable to
a denial of service attack whereby a remote peer can cause a user to insert
an unbounded number of streams into the priority tree, eventually consuming
all available memory.
This version adds a ``TooManyStreamsError`` exception that is raised when
too many streams are inserted into the priority tree. It also adds a keyword
argument to the priority tree, ``maximum_streams``, which limits how many
streams may be inserted. By default, this number is set to 1000.
Implementations should strongly consider whether they can set this value
lower.
1.1.1 (2016-05-28)
------------------
**Bugfixes**
- 2.5x performance improvement by swapping from ``queue.PriorityQueue`` to
``heapq``.
1.1.0 (2016-01-08)
------------------
**API Changes**
- Throw ``DuplicateStreamError`` when inserting a stream that is already in the
tree.
- Throw ``MissingStreamError`` when reprioritising a stream that is not in the
tree.
1.0.0 (2015-12-07)
------------------
- Initial release.
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
|
