Reimplement HTTPFields on top of a simple Array

[fabianfett]

Motivation

We like faster and simpler code. When optimizing performance we focus on often used use-cases.

Changes

• Reimplement HTTPFields on top of a simple Array

Result

• HTTPFields building becomes significantly faster
• HTTPFields lookups are faster for small fields, and only slightly slower for larger (nearly always offset by reduced build time)

Follow ups:

• Make the equality implementation smarter
• For headers that are allowed just once, exit early

[fabianfett]

Scaling results

Mallocs (mallocCountTotal)

Benchmark	N=8	N=16	N=32	N=64	N=128
Lookup
HTTPFields.contains-hit †	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
HTTPFields.contains-miss †	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
HTTPFields[name]-singleValuedField	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
HTTPFields[name]-miss	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
HTTPFields[name]-allCookieFieldsJoined	2 → 2	3 → 3	5 → 5	6 → 6	8 → 8
HTTPFields[values]-allCookieFields	2 → 2	3 → 3	5 → 5	7 → 7	8 → 8
HTTPFields[fields]-allCookieFields	2 → 2	3 → 3	5 → 5	7 → 7	8 → 8
Equality
HTTPFields.==-equal-sameOrder	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
HTTPFields.==-equal-differentOrder	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
HTTPFields.==-differsAt80%-sameOrder	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
HTTPFields.==-differsAt80%-differentOrder	0 → 0	0 → 0	0 → 0	0 → 0	0 → 0
Building
HTTPFields.append-buildFromNFields	9 → 4	11 → 5	12 → 6	13 → 7	14 → 8
HTTPFields(parsedTrailerFields)-decodeNFields	10 → 4	12 → 5	13 → 6	14 → 7	15 → 8
Copy-on-write mutation
HTTPFields-copyThenOverwriteExistingField	2 → 1	2 → 1	2 → 1	2 → 1	2 → 1
HTTPFields-copyThenInsertNewField	3 → 1	3 → 1	3 → 1	3 → 1	3 → 1

† per single call; the benchmark runs 1000 calls per iteration, so the raw counter was divided by 1000.

Instructions

Benchmark	N=8	N=16	N=32	N=64	N=128
Lookup
HTTPFields.contains-hit †	773.63 → 94.78	772.61 → 94.85	771.58 → 94.85	772.61 → 94.85	766.98 → 94.85
HTTPFields.contains-miss †	860.16 → 557.05	860.16 → 1060.86	860.16 → 2119.68	860.16 → 4143.10	860.16 → 8183.81
HTTPFields[name]-singleValuedField	7607 → 7055	7571 → 7567	7599 → 8327	7603 → 9775	7631 → 12647
HTTPFields[name]-miss	1953 → 1528	1954 → 1991	1954 → 3211	2010 → 5447	1981 → 9743
HTTPFields[name]-allCookieFieldsJoined	13647 → 13471	21631 → 21727	64191 → 64703	171135 → 172543	386303 → 389119
HTTPFields[values]-allCookieFields	7035 → 6271	10295 → 9791	24559 → 24383	58559 → 58495	122367 → 122687
HTTPFields[fields]-allCookieFields	5715 → 4795	7847 → 7139	14735 → 14279	28223 → 27743	50719 → 50687
Equality
HTTPFields.==-equal-sameOrder	5171 → 5923	8159 → 18799	10543 → 50399	11671 → 102847	14063 → 207615
HTTPFields.==-equal-differentOrder	5127 → 5891	8131 → 18639	10543 → 50431	11727 → 102783	14063 → 207615
HTTPFields.==-differsAt80%-sameOrder	4711 → 5563	4735 → 16575	7927 → 18767	8887 → 36255	10775 → 71103
HTTPFields.==-differsAt80%-differentOrder	4743 → 5559	4771 → 16175	7951 → 18799	8919 → 36255	10775 → 71167
Building
HTTPFields.append-buildFromNFields	35007 → 10463	59583 → 15591	99455 → 28111	183551 → 55359	350207 → 108671
HTTPFields(parsedTrailerFields)-decodeNFields	42879 → 14575	67903 → 21471	109247 → 37695	195583 → 72191	367103 → 139903
Copy-on-write mutation
HTTPFields-copyThenOverwriteExistingField	8863 → 5295	10727 → 6787	16319 → 11407	29567 → 22655	55647 → 45311
HTTPFields-copyThenInsertNewField	12247 → 5623	14295 → 7199	20015 → 12215	33215 → 24271	59423 → 48639

† per single call; the benchmark runs 1000 calls per iteration, so the raw counter was divided by 1000.

Retains

Benchmark	N=8	N=16	N=32	N=64	N=128
Lookup
HTTPFields.contains-hit †	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00
HTTPFields.contains-miss †	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00
HTTPFields[name]-singleValuedField	6 → 5	6 → 5	6 → 5	6 → 5	6 → 5
HTTPFields[name]-miss	3 → 2	3 → 2	3 → 2	3 → 2	3 → 2
HTTPFields[name]-allCookieFieldsJoined	14 → 13	22 → 21	70 → 69	198 → 197	454 → 453
HTTPFields[values]-allCookieFields	6 → 4	8 → 6	20 → 18	52 → 50	116 → 114
HTTPFields[fields]-allCookieFields	6 → 4	8 → 6	20 → 18	52 → 50	116 → 114
Equality
HTTPFields.==-equal-sameOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
HTTPFields.==-equal-differentOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
HTTPFields.==-differsAt80%-sameOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
HTTPFields.==-differsAt80%-differentOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
Building
HTTPFields.append-buildFromNFields	15 → 6	29 → 10	81 → 34	209 → 98	465 → 226
HTTPFields(parsedTrailerFields)-decodeNFields	23 → 8	37 → 12	89 → 36	217 → 100	473 → 228
Copy-on-write mutation
HTTPFields-copyThenOverwriteExistingField	8 → 5	10 → 7	22 → 19	54 → 51	118 → 115
HTTPFields-copyThenInsertNewField	9 → 5	11 → 7	23 → 19	55 → 51	119 → 115

† per single call; the benchmark runs 1000 calls per iteration, so the raw counter was divided by 1000.

Releases

Benchmark	N=8	N=16	N=32	N=64	N=128
Lookup
HTTPFields.contains-hit †	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00
HTTPFields.contains-miss †	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00	1.00 → 0.00
HTTPFields[name]-singleValuedField	6 → 5	6 → 5	6 → 5	6 → 5	6 → 5
HTTPFields[name]-miss	4 → 3	4 → 3	4 → 3	4 → 3	4 → 3
HTTPFields[name]-allCookieFieldsJoined	16 → 15	28 → 27	102 → 101	295 → 294	681 → 680
HTTPFields[values]-allCookieFields	10 → 8	13 → 11	27 → 25	61 → 59	126 → 124
HTTPFields[fields]-allCookieFields	9 → 7	12 → 10	26 → 24	60 → 58	125 → 123
Equality
HTTPFields.==-equal-sameOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
HTTPFields.==-equal-differentOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
HTTPFields.==-differsAt80%-sameOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
HTTPFields.==-differsAt80%-differentOrder	4 → 2	4 → 2	4 → 2	4 → 2	4 → 2
Building
HTTPFields.append-buildFromNFields	26 → 11	42 → 16	95 → 41	224 → 106	481 → 235
HTTPFields(parsedTrailerFields)-decodeNFields	49 → 25	65 → 30	118 → 55	247 → 120	504 → 249
Copy-on-write mutation
HTTPFields-copyThenOverwriteExistingField	13 → 7	15 → 9	27 → 21	59 → 53	123 → 117
HTTPFields-copyThenInsertNewField	15 → 7	17 → 9	29 → 21	61 → 53	125 → 117

† per single call; the benchmark runs 1000 calls per iteration, so the raw counter was divided by 1000.

Summary

A header field list is built once per message and then looked up by name a handful of times.

Building is 3.2–3.8× cheaper in instructions and takes about half the allocations at every size
(N=16: 11 → 5 mallocs, 59583 → 15591 instructions, 29 → 10 retains). Lookups got slower in instructions alone — every lookup benchmark keeps its malloc, retain and release counts flat or better — and only when the name is absent or the list is long.

Dividing the instructions saved on building by the extra cost the regressed lookups, gives how
many such lookups a single message can afford before the trade turns negative:

N	Instructions saved building	Affordable contains misses	Affordable [name] misses	Affordable single-valued hits
8	24544	∞ (faster)	∞ (faster)	∞ (faster)
16	43992	219	1189	∞ (faster)
32	71344	57	57	98
64	128192	39	37	59
128	241536	33	31	48

I assume that a framework/application does 5–20 by-name lookups per request. The budget covers that at every size.

Comparing two field lists in full, or using one as a dictionary key, is something real applications do rarely. Because of this going from 14063 → 207615 instructions at N=128 for equality checks is a price I am willing to pay here.

[fabianfett]

This is not sufficient. But this is known, we don't regress here: #137

Created an issue for this: #139

[glbrntt]

can we call this index?

[glbrntt]

nit: can we generalize and use != self.fields.endIndex? and in the loop can we do self.fields.formIndex(after: &self.position)?

Array is a special case where indexing is zero-based integers but I think it's good practice to use the indexing APIs as designed...

[fabianfett]

yes. that impl. is a mouthful. I have a local copy that does firstIndex(where: {}) on a slice. Need to verify that is semantically the same.

[glbrntt]

Great; it'd be good to know details because as posed it's currently a large regression!

[glbrntt]

OOI: what's your plan for making this cheaper?

[fabianfett]

If headers are equal the order is likely equal as well. This would allow us to compare in lock step, until we can't compare in lock step anymore.

Worst case we always have to go back to what we have today.

[glbrntt]

Right, I'm concerned about the unhappy path which has become quadratic in this PR.

Can you bundle in your planned changes to the equality checking here as well? Or the benchmark numbers from planned changes?

I'm a little hesitant about merging in a large regression with the promise of future improvements.

[fabianfett]

Okay added a commit that makes equality much faster, if the headers are in the same order:

  if lhs.fields.count != rhs.fields.count {
      return false
  }
  // Fast path: field lists that were built the same way carry their fields in the same
  // order, so a single lock step walk usually settles it. Element wise equality is
  // sufficient, but not necessary, for the definition above, so a mismatch only means the
  // general comparison below has to run.
  if lhs.fields.elementsEqual(rhs.fields) {
      return true
  }

Measured (instructions, p50, release, HTTPFields.==-equal-sameOrder):

  ┌─────┬────────┬───────┬─────────┐
  │  N  │ before │ after │ speedup │
  ├─────┼────────┼───────┼─────────┤
  │ 8   │ 5,943  │ 2,034 │ 2.9×    │
  ├─────┼────────┼───────┼─────────┤
  │ 16  │ 19K    │ 3,563 │ 5.3×    │
  ├─────┼────────┼───────┼─────────┤
  │ 32  │ 50K    │ 6,983 │ 7.2×    │
  ├─────┼────────┼───────┼─────────┤
  │ 64  │ 103K   │ 15K   │ 6.9×    │
  ├─────┼────────┼───────┼─────────┤
  │ 128 │ 208K   │ 30K   │ 6.9×    │
  └─────┴────────┴───────┴─────────┘

[guoye-zhang]

Doing a sort+compare should take O(n * log(n))