langcheck.metrics.metric_value

class langcheck.metrics.metric_value.MetricValue(metric_name: str, metric_values: List[NumericType], prompts: List[str] | None, generated_outputs: List[str] | tuple[List[str], List[str]] | None, reference_outputs: List[str] | None, sources: List[str] | tuple[List[str] | None, List[str] | None] | None, explanations: List[str | None] | None, language: str | None)

Bases: Generic[NumericType]

A rich object that is the output of any langcheck.metrics function.

all() → bool

Equivalent to all(metric_value.metric_values). This is mostly useful for binary metric functions.

any() → bool

Equivalent to any(metric_value.metric_values). This is mostly useful for binary metric functions.

explanations: List[str | None] | None

generated_outputs: List[str] | tuple[List[str], List[str]] | None

histogram(jupyter_mode: str = 'inline') → None

Shows an interactive histogram of all data points in MetricValue. Intended to be used in a Jupyter notebook.

This is a convenience function that calls langcheck.plot.histogram().

property is_pairwise: bool

language: str | None

metric_name: str

metric_values: List[NumericType]

prompts: List[str] | None

reference_outputs: List[str] | None

scatter(jupyter_mode: str = 'inline') → None

Shows an interactive scatter plot of all data points in MetricValue. Intended to be used in a Jupyter notebook.

This is a convenience function that calls langcheck.plot.scatter().

sources: List[str] | tuple[List[str] | None, List[str] | None] | None

to_df() → DataFrame

Returns a DataFrame of metric values for each data point.

class langcheck.metrics.metric_value.MetricValueWithThreshold(metric_name: str, metric_values: List[NumericType], prompts: List[str] | None, generated_outputs: List[str] | tuple[List[str], List[str]] | None, reference_outputs: List[str] | None, sources: List[str] | tuple[List[str] | None, List[str] | None] | None, explanations: List[str | None] | None, language: str | None, threshold: float | int, threshold_op: str)

Bases: MetricValue

A rich object that is the output of comparing an MetricValue object, e.g. metric_value >= 0.5.

all() → bool

Returns True if all data points pass the threshold.

any() → bool

Returns True if any data points pass the threshold.

property pass_rate: float

Returns the proportion of data points that pass the threshold.

threshold: float | int

threshold_op: str

property threshold_results: List[bool]

Returns a list of booleans indicating whether each data point passes the threshold.

to_df() → DataFrame

Returns a DataFrame of metric values for each data point.