Skip to content

MLECO-6480: Support for async/cancellable getNextToken

Jerry Collins requested to merge feature/cancellable_get_next_token into main
  1. Added support for new cancellable token retrival mechanism
  2. Updated backends to support new API
  3. Removed buffer in LLM class

Signed-off-by: Jerry Collins jerry.collins@arm.com Change-Id: I893ea6685e50eab53e7098e2a25a10a793af028b

Merge request reports