HttpRequestException: Destination is too short. (Parameter 'destination')
#78516
Comments
|
Tagging subscribers to this area: @dotnet/ncl Issue DetailsIs there an existing issue for this?
Describe the bugWhile using YARP with HTTP3 enabled via an Note that in this particular request, it was supposed to download the I have only encountered this error while downloading this particular file if Expected BehaviorExpected the request to complete successfully as it does for most cases, not once did it succeed for this script however. Steps To Reproduce
public class Http3ProxyRequestTransformer : HttpTransformer
{
public override async ValueTask TransformRequestAsync(
HttpContext httpContext,
HttpRequestMessage proxyRequest,
string destinationPrefix)
{
await base.TransformRequestAsync(httpContext, proxyRequest, destinationPrefix);
// Configures all requests messages to use HTTP3
proxyRequest.Version = new Version(3, 0);
proxyRequest.VersionPolicy = HttpVersionPolicy.RequestVersionOrLower;
}
}
webBuilder.ConfigureKestrel(serverOptions =>
{
serverOptions.ConfigureHttpsDefaults(httpsOptions =>
{
httpsOptions.ServerCertificateSelector = (_, name) =>
{
return name is not null
? CertificateLoader.LoadFromStoreCert(
"dev.localhost",
"My",
StoreLocation.CurrentUser,
false)
: null;
};
});
serverOptions.ListenAnyIP(5001, listenOptions =>
{
listenOptions.UseHttps();
listenOptions.Protocols = HttpProtocols.Http1AndHttp2AndHttp3;
});
});
var httpClient = new HttpMessageInvoker(new SocketsHttpHandler
{
UseProxy = false,
AllowAutoRedirect = false,
AutomaticDecompression = DecompressionMethods.None,
UseCookies = false
});
var requestConfig = new ForwarderRequestConfig
{
ActivityTimeout = TimeSpan.FromSeconds(100)
};
app.UseEndpoints(endpoints =>
{
endpoints.Map("/{**catch-all}", async httpContext =>
{
ForwarderError error = httpContext.Request.Host.ToString().Split('.') switch
{
[...]
_ => await forwarder.SendAsync(
httpContext,
Intranet,
httpClient,
requestConfig,
new Http3ProxyRequestTransformer())
};
});
});Exceptions (if any).NET Version7.0.100 Anything else?
|
|
Triage: Looks like It would be nice to get an isolated repro though. I assume this could be reproduced without reverse proxy, blazor app etc. |
|
I tried reproducing it without the blazor app, by serving just the static |
Might be because browsers don't like self-signed certs in QUIC (TLS is part of QUIC itself and the browser cannot let you manually accept the seemingly insecure cert as that's done one layer bellow). What could work, is having |
|
It very likely depends on actual value and sequence. If the flow or headers are different you may not trigger it. |
|
I was able to reproduce the error without the need of a proxy, with the help of the default Blazor Hosted Template and the following code: using var httpClient = new HttpClient();
await GetFileAsync("/_framework/blazor.webassembly.js");
async Task GetFileAsync(string relativeUri)
{
Console.WriteLine("Requesting file...");
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri(new Uri("https://localhost:5001"), relativeUri),
Version = new Version(3, 0),
VersionPolicy = HttpVersionPolicy.RequestVersionExact,
};
request.Headers.Add("accept", new[]
{
"text/html",
"application/xhtml+xml",
"application/xml;q=0.9",
"image/webp",
"image/apng",
"*/*;q=0.8",
"application/signed-exchange;v=b3;q=0.9"
});
request.Headers.Add("accept-encoding", new[] { "gzip", "defalte", "br" });
request.Headers.Add("accept-language", new[] { "en-US", "en;q=0.9", "pt;q=0.8", "es;q=0.7" });
request.Headers.Add("cache-control", "no-cache");
request.Headers.Add("pragma", "no-cache");
request.Headers.Add("sec-ch-ua", new[]
{
@"""Not_A Brand"";v=""99""",
@"""Microsoft Edge"";v=""109""",
@"""Chromium"";v=""109"""
});
request.Headers.Add("sec-ch-ua-mobile", "?0");
request.Headers.Add("sec-ch-ua-platform", @"""Windows""");
request.Headers.Add("sec-fetch-dest", "document");
request.Headers.Add("sec-fetch-mode", "navigate");
request.Headers.Add("sec-fetch-site", "none");
request.Headers.Add("sec-fetch-user", "?1");
request.Headers.Add("upgrade-insecure-requests", "1");
request.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.0.0");
HttpResponseMessage response = (await httpClient.SendAsync(
request,
new CancellationTokenSource().Token))
.EnsureSuccessStatusCode();
Console.WriteLine($"File {relativeUri.Split("/")[^1]} received with length {(await response.Content.ReadAsStringAsync()).Length}.");
}It always breaks on the I should point out that I tried serving the afore mentioned Blazor static file on its own, but wasn't able to reproduce the error, that is why I still went with a Blazor App. |
I did, and from what I could gather it happens with the request.Headers.Add("accept-encoding", new[] { "gzip" });The exception is thrown at // If a header range was set, but the value was not in the data, then copy the range
// to the name buffer. Must copy because the data will be replaced and the range
// will no longer be valid.
if (_headerNameRange != null)
{
EnsureStringCapacity(ref _headerNameOctets, _stringLength, existingLength: 0);
_headerName = _headerNameOctets;
ReadOnlySpan<byte> headerBytes = data.Slice(_headerNameRange.GetValueOrDefault().start, _headerNameRange.GetValueOrDefault().length);
headerBytes.CopyTo(_headerName);
_headerNameLength = headerBytes.Length;
_headerNameRange = null;
}However I am getting a lot of errors on VS telling me that it could not obtain the value of the local variable, so I wasn't able to learn much about the spans themselves. |
|
After messing around with my breakpoints, and if I am not mistaken, |
|
I got VS to decompile the source code and here's what I could find:
55 11 98 108 97 122 111 114 45 101 110 118 105 114 111 110 109 101 110 116 11 68 101 118 101Comprised above is one header name and the first four bytes of its value, which would eventually become Under the case 2:
_Huffman = (b & LiteralHeaderFieldWithoutNameReferenceHuffmanMask) != 0;
prefixInt = b & LiteralHeaderFieldWithoutNameReferencePrefixMask;
if (_integerDecoder.BeginTryDecode((byte)prefixInt, LiteralHeaderFieldWithoutNameReferencePrefix, out intResult))
{
if (intResult == 0)
{
throw new QPackDecodingException(SR.Format(SR.net_http_invalid_header_name, ""));
}
OnStringLength(intResult, State.HeaderName);
ParseHeaderName(data, ref currentIndex, handler);
}
else
{
_state = State.HeaderNameLength;
ParseHeaderNameLength(data, ref currentIndex, handler);
}
break;Here, at Then, at Once at Now that we've gone through the entire buffer, we fall into the following condition: if (_headerNameRange.HasValue)
{
EnsureStringCapacity(ref _headerNameOctets, _stringLength, 0);
_headerName = _headerNameOctets;
ReadOnlySpan<byte> headerBytes = data.Slice(_headerNameRange.GetValueOrDefault().start, _headerNameRange.GetValueOrDefault().length);
headerBytes.CopyTo(_headerName);
_headerNameLength = headerBytes.Length;
_headerNameRange = null;
}We've last filled I haven't had the time to dive deep into HTTP3 header encoding, so I am not able to pin-point the issue here. However, I find it odd that we don't create the header name given that we already know its value, instead we created a "range buffer". Had this range not been initialized (or been set back to |
Now that I've dug a little deeper, I assume that adding this header to the request message is just a coincidence that makes the server add its matching |
|
Sorry for the lengthy explanation, it is much more for my understanding and sake of mind than anything else. |
|
@BrunoBlanes please don't apologize, this is great investigation. You gave us stable repro and even dived into investigating this, that will help us a lot! Thank you! |
MihaZupan
added
the
tenet-reliability
|
Little update, I decided I am in too deep and began trying to fix it myself, as a learning experience since I haven't messed with the .NET source code before. As such, I came up with the following test for this scenario: using System.Linq;
using System.Net.Http.QPack;
using System.Net.Http.Unit.Tests.HPack;
using System.Text;
using Xunit;
namespace System.Net.Http.Unit.Tests.QPack
{
public class QPackDecoderTest
{
private const int MaxHeaderFieldSize = 8190;
// 4.5.6 - Literal Field Without Name Reference - (literal-header-field)
private static readonly byte[] _literalFieldWithoutNameReference = new byte[] { 0x37, 0x0d, 0x6c, 0x69, 0x74, 0x65, 0x72, 0x61, 0x6c, 0x2d, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x2d, 0x66, 0x69, 0x65, 0x6c, 0x64 };
private const string _headerNameString = "literal-header-field";
private const string _headerValueString = "should-not-break";
private static readonly byte[] _headerValueBytes = Encoding.ASCII.GetBytes(_headerValueString);
private static readonly byte[] _headerValue = new byte[] { (byte)_headerValueBytes.Length }
.Concat(_headerValueBytes)
.ToArray();
private readonly QPackDecoder _decoder;
private readonly TestHttpHeadersHandler _handler = new TestHttpHeadersHandler();
public QPackDecoderTest()
{
_decoder = new QPackDecoder(MaxHeaderFieldSize);
}
[Fact]
public void DecodesLiteralField_WithoutNameReferece()
{
// The key take away here is that the header name length should be bigger than 16 bytes
// and the header value length less than or equal to 16 bytes and they cannot all be
// read at once, they must be broken into separate buffers
byte[] encoded = _literalFieldWithoutNameReference
.Concat(_headerValue)
.ToArray();
_decoder.Decode(new byte[] { 0, 0 }, endHeaders: false, handler: _handler);
_decoder.Decode(encoded[..^7], endHeaders: false, handler: _handler);
_decoder.Decode(encoded[^7..], endHeaders: true, handler: _handler);
Assert.Equal(1, _handler.DecodedHeaders.Count);
Assert.Equal(_headerNameString, _handler.DecodedHeaders.Keys.First());
Assert.Equal(_headerValueString, _handler.DecodedHeaders.Values.First());
}
}
} |
ghost
added
the
in-pr
|
The same error occurs even when PUT is executed several times in the console program. HttpClient client = new()
{
DefaultRequestVersion = HttpVersion.Version30,
DefaultVersionPolicy = HttpVersionPolicy.RequestVersionExact
};
//....//
foreach (AasAndSubmodels item in list)
{
await _httpClient.PutAsJsonAsync($"shells/{item.AAS.IdShort}", item.AAS);
} |
ghost
added
in-pr
ghost
removed
the
in-pr
ghost
added
the
in-pr
ghost
removed
the
in-pr
ghost
locked as resolved and limited conversation to collaborators
Is there an existing issue for this?
Describe the bug
While using YARP with HTTP3 enabled via an
HttpTransformerthe request fails with aSystem.ArgumentException: Destination is too short. (Parameter 'destination').Note that in this particular request, it was supposed to download the
_framework/blazor.webassembly.jsand a couple of CSS files were successfully downloaded before:I have only encountered this error while downloading this particular file if
HttpRequestMessage.Versionis set to3.0.Expected Behavior
Expected the request to complete successfully as it does for most cases, not once did it succeed for this script however.
Steps To Reproduce
ConfigureKestreland a self-signed certificate:Exceptions (if any)
.NET Version
7.0.100
Anything else?
The text was updated successfully, but these errors were encountered: