Statistics Logbook

EvoLP includes a Logbook type which can be used to log statistics during runs.

EvoLP.Logbook — Type

Logbook()
Logbook(S::LittleDict)

A log for statistics intended for use on every iteration of an algorithm. The logbook is constructed from a LittleDict ordered dictionary which maps stat names (strings) to callables, such that statname $i$ can be computed from callable $i$.

The resulting Logbook contains:

S::LittleDict: The ordered dict of stat names and callables
records::AbstractVector: A vector of NamedTuples where each field is a statistic.

If no argument is passed, the logbook is constructed with a set of commonly statistics such as minimum, mean, median, maximum and standard deviation; in that order.

source

The Logbook receives an OrderedCollections.jl LittleDict (ordered dictionary for a small number of items) with the following format:

LittleDict("statname"::String => callable::Function)

For example, using some of the Statistics built-in functions:

julia> statnames = ["mean_eval", "max_f", "min_f", "median_f"];

julia> fns = [mean, maximum, minimum, median];

julia> thedict = LittleDict(statnames, fns)
LittleDict{String, Function, Vector{String}, Vector{Function}} with 4 entries:
  "mean_eval" => mean
  "max_f"     => maximum
  "min_f"     => minimum
  "median_f"  => median

Then the logbook can be constructed:

julia> thelogger = Logbook(thedict)
Logbook(LittleDict{AbstractString, Function, Vector{AbstractString}, Vector{Function}}("mean_eval" => Statistics.mean, "max_f" => maximum, "min_f" => minimum, "median_f" => Statistics.median), NamedTuple{(:mean_eval, :max_f, :min_f, :median_f)}[])

If no LittleDict is provided, then the logbook includes a default set of descriptive statistics: minimum, mean, median, maximum and standard deviation—in that order.

Computing statistics

After instantiating the Logbook, you can use the compute! function on each iteration of an algorithm. The statistics are stored in the records field inside the Logbook, which is a vector of records (NamedTuples). This makes it easier to export as a DataFrame.

EvoLP.compute! — Function

compute!(logger::Logbook, data::AbstractVector)
compute!(notebooks::Vector{Logbook}, data::Vector{AbstractVector})

Computes statistics for logger (or a vector of loggers) using data, which is usually a vector of fitnesses. All calculations are done in place, so the logger records will be updated.

source

The compute! function can be called either by providing a logbook to update, or a vector of Logbooks. This is useful if that which you want to calculate depends on different data sources (e.g. some statistics are computed from fitness while some others use the population, etc.)

Statistics at a glance

If you prefer to have a quick overview of your Logbook, you can do so using the summarise function:

EvoLP.summarise — Function

summarise(logger::Logbook)
summarise(notebooks::Logbook)

Print and plot descriptive statistics for a given logger (or a vector of loggers).

source

summarise will go through each of the statistics and present a summary and a Unicode plot:

 mean_f 
max: -0.5333333333333333 
avg: -5.6193333333333335
median: -5.716666666666667 
min: -7.366666666666666
std: 1.0150625419575374
             ┌────────────────────────────────────────┐ 
           0 │▖                                       │ 
             │▐                                       │ 
             │▗                                       │ 
             │▐                                       │ 
             │ ▌                                      │ 
             │ ▌                                      │ 
             │ ▐▖                                     │ 
   mean_f    │  ▌                              ▐      │ 
             │  ▌         ▐  ▖                ▖▞▖     │ 
             │  ▐▄▖   ▗  ▖▐▌▖▙▐    █▌    ▗▖   █▌▌ ▖▗▚ │ 
             │  ▐▜█▗▖▙▛▟▐▜▘▙▜▐▐▖ █▌▌▚ ▖▗▗▌█ ▖▗█▌▌▐▙▐▐ │ 
             │   ▐▌███ █▌  █▝ ▘▙▟▝█▘▐▐▜▌▛▌██▀█▌▌▐▐▘▀ ▙│ 
             │   ▝▌▝▛█ ▝▘  ▜    ▘ ▐ ▝▌ ▘  ▘▜ ▀▘  ▀   █│ 
             │       ▐     ▐      ▝                   │ 
          -8 │                                        │ 
             └────────────────────────────────────────┘ 
              0                                    100  
                                 it