deepluq.metrics_vla
===================

.. py:module:: deepluq.metrics_vla


Classes
-------

.. autoapisummary::

   deepluq.metrics_vla.TokenMetrics
   deepluq.metrics_vla.OutputMetrics


Module Contents
---------------

.. py:class:: TokenMetrics

   .. py:attribute:: shannon_entropy_list
      :value: []



   .. py:attribute:: token_prob
      :value: []



   .. py:attribute:: pcs
      :value: []



   .. py:attribute:: token_prob_inv
      :value: []



   .. py:attribute:: pcs_inv
      :value: []



   .. py:attribute:: deepgini
      :value: []



   .. py:method:: calculate_metrics(logits)


   .. py:method:: compute_norm_inv_token_metrics(logits)

      Compute various token-level uncertainty and confidence metrics from model logits,
      normalize them to [0, 1], and invert selected metrics so that higher values
      consistently indicate greater uncertainty.

      Metrics computed:
      - Shannon Entropy (normalized): uncertainty measure normalized by log2(num_classes).
      - Max Token Probability (normalized and inverted): confidence of top predicted token,
        normalized and inverted so higher means less confidence.
      - PCS (Prediction Confidence Score) (inverted): difference between top two token probabilities,
        inverted so higher means more uncertainty.
      - DeepGini (normalized): uncertainty measure normalized by its max possible value.

      :param logits: raw output logits from the model with shape (batch_size, num_classes).
      :type logits: torch.Tensor

      :returns:

                four lists of float values rounded to 5 decimals, corresponding to:
                      [shannon_entropy, max_token_prob_inverted, pcs_inverted, deepgini]
      :rtype: list



   .. py:method:: clear()


.. py:class:: OutputMetrics

   Compute various instability and variability metrics for robot actions and TCP positions.

   Author: Pablo Valle
   Time  : 05/22/2025


   .. py:attribute:: VARIABILITY
      :value: 4



   .. py:method:: _action_array(actions: List[Dict[str, Any]]) -> numpy.ndarray
      :staticmethod:


      Convert a list of action dicts to a NumPy array.

      Each action dict should contain:
      - "world_vector"
      - "rot_axangle"
      - "gripper"



   .. py:method:: _compute_instability(arr: numpy.ndarray, order: int = 1, scale: float = 1.0) -> numpy.ndarray
      :staticmethod:


      Compute instability metrics by taking successive differences.

      :param arr: Input array of shape (T, M).
      :type arr: np.ndarray
      :param order: Number of differences to compute (1=position, 2=velocity, 3=acceleration).
      :type order: int
      :param scale: Scaling factor for difference magnitude.
      :type scale: float

      :returns: Instability per dimension (M,).
      :rtype: np.ndarray



   .. py:method:: compute_position_instability(actions: List[Dict[str, Any]]) -> numpy.ndarray


   .. py:method:: compute_velocity_instability(actions: List[Dict[str, Any]]) -> numpy.ndarray


   .. py:method:: compute_acceleration_instability(actions: List[Dict[str, Any]]) -> numpy.ndarray


   .. py:method:: _tcp_array(poses: List[List[float]]) -> numpy.ndarray
      :staticmethod:


      Extract TCP positions (x, y, z) from poses.



   .. py:method:: compute_TCP_position_instability(poses: List[List[float]]) -> numpy.ndarray


   .. py:method:: compute_TCP_velocity_instability(poses: List[List[float]]) -> numpy.ndarray


   .. py:method:: compute_TCP_acceleration_instability(poses: List[List[float]]) -> numpy.ndarray


   .. py:method:: compute_TCP_jerk_instability_gradient(poses: List[List[float]]) -> numpy.ndarray

      Compute TCP jerk using numerical gradients and return jerk magnitude per time step.



   .. py:method:: compute_execution_variability(variability_models: List[Any], image: Any, action_space: Any, instruction: Any, obs: Dict[str, Any], model_name: str) -> numpy.ndarray
      :staticmethod:


      Compute variability across multiple models' actions.

      :returns: Standard deviation of actions across models.
      :rtype: np.ndarray