Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: consolidate azure with openai provider #60

Merged
merged 5 commits into from
Sep 27, 2024

Conversation

codefromthecrypt
Copy link
Contributor

@codefromthecrypt codefromthecrypt commented Sep 25, 2024

note: in order to be able to run just integration -k azure and not also run openai and ollama, I needed to get rid of the openai subdir in the tests (as it was before I started)

@codefromthecrypt
Copy link
Contributor Author

draft until #54

assert reply_message.content == [Text(text="Hello!")]
assert reply_usage.total_tokens == 35
mock_post.assert_called_once_with(
f"{azure_provider.client.base_url}/chat/completions?api-version={azure_provider.api_version}",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hard to see here, but it is the same openai endpoint as normal openai, but only at chat/completions. This is why the openai python SDK needs little config to switch to azure

user-agent:
- python-httpx/0.27.2
method: POST
uri: https://test.openai.azure.com/openai/deployments/test-azure-deployment/chat/completions?api-version=2024-05-01-preview
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelneale this is a scrubbed real request/response

@codefromthecrypt
Copy link
Contributor Author

@anuraaga this has two commits, but if any interest in the azurification

Copy link
Contributor

@anuraaga anuraaga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some nits, LGTM

tests/providers/conftest.py Outdated Show resolved Hide resolved
tests/providers/conftest.py Outdated Show resolved Hide resolved
This scrubs sensitive request data in provider-specific way. Note that headers
are case-sensitive!
"""
if "openai" in request.uri:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the presence of openai in the URI to indicate azure is still blowing my mind, especially so since it's not obvious the uri doesn't contain the host which could otherwise have been api.openai.com.

Perhaps request.headers["host"] == os.environ.get("AZURE_CHAT_COMPLETIONS_HOST_NAME") is an option? Admittedly not great either, just a note to see if there isn't anything to improve here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was a goof on my side, I meant to put openai.azure.com which I think we can keep as a condition . The problem with using that ENV variable is that it is weird, like it isn't a hostname, rather a base URL.

@codefromthecrypt
Copy link
Contributor Author

currently at least the azure deployment I'm using doesn't support vision even though the model seems the same as openai. For now, not adding the vision tests, but someone can try again in a subsequent PR perhaps with another deployment:

E               {
E                 "error": {
E                   "message": "Invalid content type. image_url is only supported by certain models.",
E                   "type": "invalid_request_error",
E                   "param": "messages.[4].content.[0].type",
E                   "code": null
E                 }
E               }

@codefromthecrypt
Copy link
Contributor Author

@anuraaga can you verify I am understanding right, basically that we can't use vision yet?

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision?tabs=rest#gpt-4-turbo-provisioned-managed-availability

I've tried gpt-4 and no dice, but above docs make sense

e.g. below fail before getting to assertions with invalid_request_error

@pytest.mark.vcr()
def test_azure_vision(default_azure_env):
    reply_message, reply_usage = vision(AzureProvider, AZURE_MODEL)

    assert reply_message.content == [Text(text='The first entry in the menu says "Ask Goose."')]
    assert reply_usage.total_tokens == 14241


@pytest.mark.integration
def test_azure_vision_integration():
    reply = vision(AzureProvider, AZURE_MODEL)

    assert "ask goose" in reply[0].text.lower()

Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
@codefromthecrypt codefromthecrypt marked this pull request as ready for review September 26, 2024 01:03
base_url=url,
timeout=httpx.Timeout(60 * 10),
)
ollama_url = os.environ.get("OLLAMA_HOST", OLLAMA_HOST)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@baxen lemme know if you think this is more sensible than before

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this looks good!

Copy link
Collaborator

@baxen baxen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@michaelneale michaelneale merged commit 41f4e63 into square:main Sep 27, 2024
5 checks passed
@codefromthecrypt codefromthecrypt deleted the azure-consolidate branch September 27, 2024 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants