Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement IUtf8SpanParsable on IPAddress and IPNetwork #102144

Open
wants to merge 42 commits into
base: main
Choose a base branch
from

Conversation

edwardneal
Copy link
Contributor

@edwardneal edwardneal commented May 13, 2024

Closes #103111

Relates to #81500. cc @tannergooding - hopefully this doesn't tread on any prior work.

This implements IUtf8SpanParsable on IPAddress and IPNetwork. It does so by making IPAddressParser generic between byte and char. Doing so needed deeper changes than I'd have liked (the separators aren't considered constants, which means that I had to swap a switch statement and its "goto case" statements for a set of if statements) As I did that, I replaced most of the unsafe code and pointers with ReadOnlySpans (which eliminates a pattern where downstream code was pinning an ROS and passing its pointer.)

IPAddressParser, IPv4AddressHelper and IPv6AddressHelper are shared with System.Net.Quic, System.Net.Security and System.Private.Uri. The changes to the first two projects are pretty minor, consisting of one change to TargetHostNameHelper. System.Private.Uri is similar, but it makes more extensive use of pointers.

There are two differences between this PR and the underlying issue:

  • I've added the public methods Parse(ReadOnlySpan<byte>) and TryParse(ReadOnlySpan<byte>, out IPNetwork) to IPNetwork.
  • IPNetwork implements IUtf8SpanParsable explicitly rather than implicitly.

In both cases, this is done to maintain consistency with the ISpanParsable members which were already implemented. I've taken this to be covered by the original API review.

It's worth noting that IPv6AddressHelper<TChar>.IsValid in System.Private.Uri continues to reference char* rather than ReadOnlySpan<TChar>. This is inconsistent, but I've left it alone for now. I'd prefer to address that in a follow-up PR which replaces the unsafe components from Uri.PrivateParseMinimal and the methods it calls with a ROS.

This is preparation for the work required to enable IUtf8SpanParsable support on IPAddress and IPNetwork.
Also replaced a few instances of unsafe code with spans.
This uses the same style of generic constraints as the numeric parsing in corelib (although it can't access IUtfChar<TSelf>.)
All tests pass, although the UTF8 code paths haven't yet been tested.
Some restructuring is pending as I plumb IPAddressParser to the updated generic methods.
Future commits will handle the downstream dependencies of IPvXAddressHelper - at present, this looks like Uri and TargetHostNameHelper.
This allows me to eliminate the nested class and now-redundant generic type constraints on each method
These are just the changes required to make System.Private.Uri compile.
* Split IPAddressParser and IPv6AddressHelper into multiple files and shared common components between projects.
* Updated some of the Uri-specific additions to IPvXAddressHelper files to use spans.
* Minor adjustments to project files to support the above.
Removed a using statement in the ref project, small reduction in the csproj diff for System.Net.Primitives
Copy link

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@tannergooding
Copy link
Member

Hi @edwardneal, sorry for the delay in getting to this, I had missed the tag initially since this is under the networking area.

It's worth noting that IPAddress and IPNetwork are not part of APIs that #81500 was approved for, and so before this can move forward there needs to be a new API proposal that specifically asks for implementing IUtf8SpanParsable on these types.

That proposal should be straightforward and I expect have no issues going through API review, at which point this can likely get merged. If you could assist in opening such a proposal, that should help the process go along faster.

@edwardneal
Copy link
Contributor Author

Thanks @tannergooding. Sorry for the confusion here - I read the original issue as approving the addition of IUtf8SpanParsable and IUtf8SpanFormattable to anything already implementing ISpanFormattable, and IPAddress and IPNetwork both do so. Should I just raise the API proposal for the additional IPNetwork.Parse(ReadOnlySpan<byte>) and IPNetwork.TryParse(ReadOnlySpan<byte>, out IPNetwork) methods and IPNetwork's explicit interface implementation, or should I also include IPAddress in that?

@tannergooding
Copy link
Member

should I also include IPAddress in that?

We want to include all new public APIs and interfaces in the proposal whether that is for IUtf8SpanParsable, IUtf8SpanFormattable, or both; as well as any overloads beyond this.

I read the original issue as approving the addition of IUtf8SpanParsable and IUtf8SpanFormattable to anything already implementing ISpanFormattable

The general spirit behind the feature fits that, but we still do explicit review and sign-off as there are often edge cases that we may warrant different consideration.

@karelz
Copy link
Member

karelz commented Jun 6, 2024

Given that we are blocked on API review, I would recommend to either change the PR to Draft or close it, until the API is approved. @edwardneal what do you prefer and can you please do it? Thanks!

@karelz karelz added the blocked Issue/PR is blocked on something - see comments label Jun 6, 2024
@karelz karelz requested a review from tannergooding June 6, 2024 15:00
@edwardneal edwardneal marked this pull request as draft June 6, 2024 16:42
@MihaZupan MihaZupan marked this pull request as ready for review June 14, 2024 20:33
There are now no circumstances where a string is passed to if_nametoindex or SystemNative_InterfaceNameToIndex, or the calling InterfaceNameToIndex method in InterfaceInfoPal. Removed these.
An interface name comes from a scope in a valid IPv6 address. There'll always be at least three characters (::%) in front of the scope, so taking the length of an interface name which stripping these three leading characters away and adds a single null terminator will never overflow an integer.
@ManickaP ManickaP added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Jul 26, 2024
@ManickaP
Copy link
Member

I added the NO MERGE label. Once we branch out, I'll remove it and this could be merged (mid August).

This eliminates a number of changes required to transcode from UTF8 and Unicode to ANSI.
Instead of using if_nametoindex on Windows, this now uses ConvertInterfaceNameToLuidW & ConvertInterfaceLuidToIndex.
When running in a container, Windows does not have a static loopback interface name.
Moved tests of InterfaceInfoPal to the PalTests project, rather than testing with hardcoded IP addresses with real scope IDs.
Made the tests use real (and in Windows' case, dynamic) interface names.
@liveans liveans removed the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Aug 26, 2024
@liveans
Copy link
Member

liveans commented Aug 26, 2024

Deleted NO-MERGE label since branching for .NET 9 has been done.

}
buffer[buffer.Length - 1] = '\0';

if (Interop.IpHlpApi.ConvertInterfaceNameToLuid(buffer, ref interfaceLuid) != 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this produces an error, but in the finally we then Free the native memory, will that end up corrupting whatever error code the caller might need to retrieve?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won't on Windows: the documentation doesn't say that any of the methods changes the result of GetLastError, and testing confirms this - the return value is the only success/failure response, so there's no error code to corrupt. SetLastError was set to true on the interop definitions though, I've corrected this.

It will on Unix, so I've wrapped the Free in GetLastPInvokeError/SetLastPInvokeError as per the pattern in CriticalHandle.Cleanup.

Copy link
Member

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Left a few comments/questions, but otherwise LGTM.

* SetLastError has the correct value in the Windows interop layer.
* Preserve InterfaceNameToIndex's errno in the Unix InterfaceInfoPal.
@MihaZupan
Copy link
Member

@EgorBot

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;
using System.Net;

public class Bench
{
    [Benchmark]
    [Arguments("192.168.0.1")]
    [Arguments("4294967295")]
    [Arguments("037777777777")]
    [Arguments("0xff.0x7f.0x20.0x01")]
    [Arguments("192.168.0.0/16")]
    [Arguments("::192.168.0.1")]
    [Arguments("100:0:1:2:0:0:000:abcd")]
    [Arguments("Fe08::1%13542")]
    [Arguments("1:2:3:4:5:6:7:8::")]
    public bool TryParse(string s) => IPAddress.TryParse(s, out _);

    [Benchmark]
    [Arguments("http://192.168.0.1:123/foo")]
    [Arguments("http://[100:0:1:2:0:0:000:abcd]:123/foo")]
    public string UriHost(string s) => new Uri(s).Host;
}

@edwardneal
Copy link
Contributor Author

edwardneal commented Oct 6, 2024

Thanks MihaZupan. That's quite a wide regression in ::192.168.0.1, and I don't see a clear reason why. Some performance is left on the table in HexConverter.FromChar and similar (it doesn't cast its input to a uint, so the JIT doesn't elide the bounds check and we end up performing it twice) but we were using that method anyway - just indirectly, as a result of calling Uri.FromHex. In any event though, this method isn't called in the slower case - that test case's code path hasn't actually changed.

I've reverted the only other difference so that there's a common baseline, and I'll keep looking at this. I'm faintly suspicious that passing around an int rather than a char is causing the JIT to implicitly reintroduce bounds checks, but I wouldn't expect them to have such a large impact in this case.

@EgorBo
Copy link
Member

EgorBo commented Oct 6, 2024

Some performance is left on the table in HexConverter.FromChar and similar (it doesn't cast its input to a uint, so the JIT doesn't elide the bounds check and we end up performing it twice)

Can you point me exactly to the non-elided bound check? Last I checked, all users of FromChar give it something JIT can see it's never negative. E.g. Uri.FromHex passes char to HexConverter.FromChar, hence, never negative.

@edwardneal
Copy link
Contributor Author

I agree. FromChar is defined as

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static int FromChar(int c)
{
    return c >= CharToHexLookup.Length ? 0xFF : CharToHexLookup[c];
}

c isn't cast to an unsigned type before the comparison to CharToHexLookup.Length, so if the caller passes it an int then a bounds check is performed.

That's not normally a problem - like you said, Uri.FromHex passes it a char, so when it's inlined it makes perfect sense that the JIT can see which datatype is really being passed around. I'm passing it an integer though, so the bounds check would likely remain.

I'm hesitant to say that it's the only cause of the performance regression because when I benchmarked it, the bounds checks only introduced a 0.2ns/call performance gap, even on a slower laptop. I'm going to test the idea over the next few days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-System.Net community-contribution Indicates that the PR has been added by a community member new-api-needs-documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[API Proposal]: Implement IUtf8SpanParsable on IPAddress & IPNetwork