perfume package¶
Submodules¶
perfume.analyze module¶
perfume.analyze
contains transformation and analysis tools.
These functions mostly take as input the samples collected by
perfume.bench()
.
-
perfume.analyze.
bucket_resample_timings
(samples, sample_size=10, agg=<function mean>, sample_count=1000)[source]¶
-
perfume.analyze.
cumulative_quantiles
(samples, rng=None)[source]¶ Computes “cumulative quantiles” for each function.
That is, for each time, what are the extremes, median, and 25th/75th percentiles for all observations up until that point.
-
perfume.analyze.
cumulative_quantiles_plot
(samples, plot_width=960, plot_height=480, show_samples=True)[source]¶ Plots the cumulative quantiles along with a scatter plot of observations.
-
perfume.analyze.
isolate
(samples)[source]¶ For each function, isolates its begin and end times.
Within each function’s begins and ends, each begin will be equal to the previous end. This gives a sequence of begins and ends as if each function were run in isolation with no benchmarking overhead.
-
perfume.analyze.
ks_test
(t)[source]¶ Runs the Kolmogorov-Smirnov test across functions.
Returns a DataFrame containing all pairwise K-S test results.
The standard K-S test computes \(D\), which is the maximum difference between the empirical CDFs.
The result value we return is the \(Z\) value, defined as
\[Z = D / \sqrt{(n + m) / nm}\]where \(n\) and \(m\) are the respective sample sizes.
\(Z\) is typically interpreted using a lookup table, i.e. for confidence level \(\alpha\), we want to see a \(Z\) greater than \(c(\alpha)\):
\(\alpha\)
0.10
0.05
0.025
0.01
0.005
0.001
\(c(\alpha)\)
1.22
1.36
1.48
1.63
1.73
1.95
perfume.perfume module¶
Main module.
-
perfume.perfume.
bench
(*fns, samples=None, efficiency=0.9)[source]¶ Benchmarks functions, displaying results in a Jupyter notebook.
Runs
fns
repeatedly, collecting timing information, untilKeyboardInterrupt
is raised, at which point benchmarking stops and the results so far are returned.- Parameters
fns (list of callable) – A list of functions to benchmark and compare
samples (pandas.DataFrame) – Optionally, pass the results of a previous call to
bench()
to continue from its already collected data.efficiency (float) – Number between 0 and 1. Represents the target portion of time we aim to spend running the functions under test (so, we spend up to \(1 - efficiency\) time analyzing and rendering plots).
- Returns
A dataframe containing the results so far. The row index is just an autoincrement integer, and the column index is a
MultiIndex
where the first level is function name and the second level isbegin
orend
.- Return type
Module contents¶
Top-level package for perfume.
-
perfume.
bench
(*fns, samples=None, efficiency=0.9)[source]¶ Benchmarks functions, displaying results in a Jupyter notebook.
Runs
fns
repeatedly, collecting timing information, untilKeyboardInterrupt
is raised, at which point benchmarking stops and the results so far are returned.- Parameters
fns (list of callable) – A list of functions to benchmark and compare
samples (pandas.DataFrame) – Optionally, pass the results of a previous call to
bench()
to continue from its already collected data.efficiency (float) – Number between 0 and 1. Represents the target portion of time we aim to spend running the functions under test (so, we spend up to \(1 - efficiency\) time analyzing and rendering plots).
- Returns
A dataframe containing the results so far. The row index is just an autoincrement integer, and the column index is a
MultiIndex
where the first level is function name and the second level isbegin
orend
.- Return type