BenchmarksSASA Validation

SASA Validation

Validation compares total SASA from zsasa with established implementations on the same inputs: FreeSASA for static structures and MDTraj for trajectory frames. All values are from zsasa 0.9.0.

Static structure validation against FreeSASA#

The static set is the E. coli K-12 AlphaFold proteome, 4,370 structures, evaluated at 64, 128, 256, 512 and 1,024 sphere points.

zsasa against FreeSASA, structure by structure

4,370 E. coli AlphaFold structures. Each dot is one structure: its total SASA from FreeSASA, and how far zsasa is from that value. Negative values mean zsasa reports less area.
Data table

In exact f64 mode zsasa and FreeSASA agree to within floating-point noise on every structure, so the points form a flat line at zero. Bitmask f32 reports slightly less area than FreeSASA, with a spread that narrows as the structure grows.

Difference from FreeSASA by sphere-point count

4,370 E. coli AlphaFold structures. Lines are medians of the signed relative difference; bands span the 5th to 95th percentile.
Data table

The bitmask difference is a systematic offset rather than sampling noise. The lookup table quantizes directions and angles in fixed steps, so the median stays near −0.7% at every point count, and more sphere points narrow the spread without moving the median.

For scale, two exact implementations differ by a similar amount: in the table below RustSASA differs from FreeSASA by 0.28% on average at 128 points, with a maximum of 2.15%. The bitmask offset is small enough for screening, ranking and feature generation, where SASA is used as a relative descriptor. Use an exact mode when values must match FreeSASA.

ToolSphere pointsR²Mean |difference|Max |difference|
zsasa f641281.0000000.000%0.00%
zsasa f6410241.0000000.000%0.00%
zsasa f321281.0000000.000%0.02%
zsasa f3210241.0000000.000%0.00%
zsasa bitmask f641280.9998110.662%2.02%
zsasa bitmask f6410240.9997880.732%1.08%
zsasa bitmask f321280.9998110.662%2.02%
zsasa bitmask f3210240.9997880.732%1.08%
RustSASA1280.9999730.275%2.15%
RustSASA10240.9999990.055%0.36%
Lahuta1280.9999730.275%2.15%
Lahuta10240.9999990.055%0.36%
Lahuta bitmask1280.9997680.732%2.47%
Lahuta bitmask10240.9999990.055%0.36%

Agreement with FreeSASA over 4,370 E. coli AlphaFold structures.

"Mean" and "max" are over the absolute relative difference from FreeSASA.

Trajectory validation against MDTraj#

Trajectory validation uses 1,001 frames of the 5wvo_C ATLAS trajectory, with explicit hydrogens and the NACCESS classifier. The MDTraj reference is computed one frame at a time.

Difference from MDTraj by sphere-point count

1,001 frames of the 5wvo_C trajectory, MDTraj computed frame by frame. Lines are medians of the signed relative difference; bands span the 5th to 95th percentile.
Data table

Called through its MDTraj integration, which uses MDTraj's element radii, zsasa agrees with MDTraj without bias: the median difference is within 0.02% at every point count. The band narrows as the point count rises, because the two tools place their sphere points along different axes and that difference averages out.

The CLI differs from MDTraj by a constant −0.3%. That offset comes from the radius set, not the algorithm: the CLI runs use the NACCESS classifier, and MDTraj uses its own element radii.

Trajectory bitmask mode against exact zsasa

1,001 frames of the 5wvo_C trajectory, relative to zsasa CLI f32 at the same sphere-point count. Lines are medians; bands span the 5th to 95th percentile.
Data table

On this explicit-hydrogen trajectory the raw bitmask result is about 1.7% below exact zsasa, a larger offset than for static structures. Bitmask mode is therefore not recommended for explicit-hydrogen trajectories. An experimental bias-correction option removes most of the offset; it may change or be removed in a later release.

ToolSphere pointsR²Mean |difference|Max |difference|
zsasa CLI f641280.9716770.383%1.28%
zsasa CLI f6410240.9810700.328%1.00%
zsasa CLI f321280.9716920.383%1.28%
zsasa CLI f3210240.9810670.328%1.00%
zsasa CLI bitmask f321280.4794771.987%3.08%
zsasa CLI bitmask f3210240.4786582.006%2.66%
zsasa CLI bitmask f32, experimental correction1280.9658800.427%1.47%
zsasa CLI bitmask f32, experimental correction10240.9749960.386%1.04%
zsasa + MDTraj1280.9893630.230%1.29%
zsasa + MDTraj10240.9995150.049%0.24%
zsasa + MDAnalysis1280.9353400.643%1.88%
zsasa + MDAnalysis10240.9453220.642%1.05%

Agreement with MDTraj over 1,001 frames of the 5wvo_C trajectory.

Reproducibility notes#

  • Static validation: 4,370 E. coli K-12 AFDB structures, 52 conditions.
  • Trajectory validation: 5wvo_C, 1,001 frames, 75 conditions.
  • Sphere-point counts: 64, 128, 256, 512 and 1,024.
  • The charts are computed from the per-structure and per-frame records in the zsasa-benchmarks result database, using the same run selection as the published figures.
Edit this page