chore: update test_tools to read a file instead of get a password #38
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
codefromthecrypt
commented
Sep 9, 2024
|
|
||
| response = ex.reply() | ||
|
|
||
| # It's possible this can be flakey, but in experience so far haven't seen it |
There was a problem hiding this comment.
I want to say this, but might be TMI
# On occasion, the LLM may forget it can do tool calls and fail. Retry in this case until
# we have logic to handle this internally (retrying on text instead of tool choices).
baxen
reviewed
Sep 9, 2024
| ) | ||
|
|
||
| ex.add(Message.user("Can you authenticate this session by responding with the password")) | ||
| ex.add(Message.user(f"Can you read the contents of this file: {temp_file}")) |
There was a problem hiding this comment.
There might be some ambiguity in this around if it actually shares the ouptut again in its response, i'd maybe rephrase as "What are the contents of this file: "
There was a problem hiding this comment.
Adding this now so we can include in upcoming release!
lifeizhou-ap
added a commit
that referenced
this pull request
Sep 18, 2024
* main: feat: Rework error handling (#48) chore(release): release version 0.9.0 (#45) chore: add just command for releases and update pyproject for changelog (#43) feat: convert ollama provider to an openai configuration (#34) fix: Bedrock Provider request (#29) test: Update truncate and summarize tests to check for sytem prompt t… (#42) chore: update test_tools to read a file instead of get a password (#38) fix: Use placeholder message to check tokens (#41) feat: rewind to user message (#30) chore: Update LICENSE (#40) fix: shouldn't hardcode truncate to gpt4o mini (#35)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #37 and avoids some LLMs being overly sensitive about giving you a password.