langcheck.metrics.text_structure#
- langcheck.metrics.text_structure.contains_all_strings(generated_outputs: list[str] | str, strings: list[str], case_sensitive: bool = False, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs contain all strings in of a given list. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
strings – A list of strings to match
case_sensitive – Whether to match case sensitively or not, default False
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.contains_any_strings(generated_outputs: list[str] | str, strings: list[str], case_sensitive: bool = False, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs contain any strings in a given list. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
strings – A list of strings to match
case_sensitive – Whether to match case sensitively or not, default to
False
.prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.contains_regex(generated_outputs: list[str] | str, regex: str, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs partially contain a given regular expression. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
regex – The regular expression to match
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.is_float(generated_outputs: list[str] | str, min: float | None = None, max: float | None = None, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs can be parsed as floating point numbers, optionally within a min/max range. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
min – The optional minimum valid float
max – The optional maximum valid float
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.is_int(generated_outputs: list[str] | str, domain: Iterable[int] | Container[int] | None = None, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs can be parsed as integers, optionally within a domain of integers like range(1, 11) or {1, 3, 5}. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
domain – The optional domain of valid integers
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.is_json_array(generated_outputs: list[str] | str, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs can be parsed as JSON arrays. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.is_json_object(generated_outputs: list[str] | str, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs can be parsed as JSON objects. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.matches_regex(generated_outputs: list[str] | str, regex: str, prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs fully match a given regular expression. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
regex – The regular expression to match
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object
- langcheck.metrics.text_structure.validation_fn(generated_outputs: list[str] | str, valid_fn: Callable[[str], bool], prompts: list[str] | str | None = None) MetricValue[int] [source]#
Checks if generated outputs are valid according to an arbitrary function. This metric takes on binary 0 or 1 values.
- Parameters:
generated_outputs – The model generated output(s) to evaluate
valid_fn – A function that takes a single string and returns a bool determining whether the string is valid or not. The function can also raise an exception on failure.
prompts – The prompts used to generate the output(s). Prompts are optional metadata and not used to calculate the metric.
- Returns:
An
MetricValue
object