Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Generator] Multipart support #366

Merged
merged 87 commits into from
Nov 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
53d0ada
WIP on multipart
czechboy0 Nov 1, 2023
04103d5
Bump OpenAPIKit to 3.0.0-rc.3 and recognize more base64/binary encodi…
czechboy0 Nov 1, 2023
3579bdc
Merge branch 'hd-openapikit-rc3' into hd-multipart
czechboy0 Nov 1, 2023
4655642
Merge branch 'main' into hd-multipart
czechboy0 Nov 1, 2023
6de5e53
Add a doc
czechboy0 Nov 1, 2023
e6fc85d
WIP
czechboy0 Nov 2, 2023
1374a58
wip
czechboy0 Nov 2, 2023
d54c702
Prepared another test operation
czechboy0 Nov 2, 2023
0a072fa
WIP
czechboy0 Nov 2, 2023
f35dc20
Add a custom part header
czechboy0 Nov 3, 2023
e0d2724
Add a second part
czechboy0 Nov 3, 2023
cf64e4a
WIP
czechboy0 Nov 3, 2023
842496a
Test with chunking
czechboy0 Nov 3, 2023
d636e40
Added an undocumented case
czechboy0 Nov 3, 2023
bd54cde
wip
czechboy0 Nov 3, 2023
0062bfe
wip
czechboy0 Nov 4, 2023
c1fadfc
Added filename support
czechboy0 Nov 4, 2023
185c112
wip
czechboy0 Nov 4, 2023
7be4532
wip
czechboy0 Nov 4, 2023
0408734
Merge branch 'main' into hd-multipart
czechboy0 Nov 6, 2023
e600e1b
wip
czechboy0 Nov 6, 2023
baaa5c0
Merge branch 'main' into hd-multipart
czechboy0 Nov 6, 2023
4cd37e1
wip
czechboy0 Nov 6, 2023
3e7d9af
wip
czechboy0 Nov 6, 2023
4d9f08d
wip
czechboy0 Nov 6, 2023
369b822
wip
czechboy0 Nov 6, 2023
674e2d7
Merge branch 'main' into hd-multipart
czechboy0 Nov 7, 2023
1250c38
wip
czechboy0 Nov 7, 2023
5f15742
wip
czechboy0 Nov 7, 2023
fb97e9e
wip
czechboy0 Nov 7, 2023
c74678f
wip
czechboy0 Nov 7, 2023
8bbcc26
wip
czechboy0 Nov 7, 2023
a22988a
wip
czechboy0 Nov 8, 2023
72a47a5
wip
czechboy0 Nov 8, 2023
148498f
wip
czechboy0 Nov 8, 2023
ca18611
wip
czechboy0 Nov 8, 2023
fd8da73
SOAR-0009 - Typesafe multipart with streaming
czechboy0 Nov 8, 2023
1a115d0
Merge branch 'hd-soar-0009' into hd-multipart
czechboy0 Nov 8, 2023
fd0131b
WIP starting on the generation logic
czechboy0 Nov 8, 2023
7daf63f
wip
czechboy0 Nov 8, 2023
3d679a1
Merge branch 'main' into hd-multipart
czechboy0 Nov 21, 2023
0b6332b
Update with the latest runtime changes
czechboy0 Nov 21, 2023
676d6de
wip
czechboy0 Nov 21, 2023
4e1ed94
wip
czechboy0 Nov 21, 2023
4ad7cf0
wip
czechboy0 Nov 21, 2023
de9c8d0
wip
czechboy0 Nov 21, 2023
687dbae
wip
czechboy0 Nov 21, 2023
572e105
wip
czechboy0 Nov 21, 2023
7249268
Remove multipart-echo
czechboy0 Nov 22, 2023
8eb637b
Remove multipart download example
czechboy0 Nov 22, 2023
6db82d0
wip
czechboy0 Nov 22, 2023
2ed6dd5
wip
czechboy0 Nov 22, 2023
5e868f1
wip
czechboy0 Nov 22, 2023
56eef0d
Basic client request side working
czechboy0 Nov 22, 2023
305d4a2
wip
czechboy0 Nov 22, 2023
21cdebd
wip
czechboy0 Nov 22, 2023
9ee6236
wip
czechboy0 Nov 22, 2023
5fbc15f
wip
czechboy0 Nov 22, 2023
99281cd
wip
czechboy0 Nov 22, 2023
a737737
wip
czechboy0 Nov 22, 2023
5b17b48
Basic multipart generation working end to end
czechboy0 Nov 22, 2023
cef4282
Add another multipart operation for testing
czechboy0 Nov 23, 2023
9066c40
wip
czechboy0 Nov 23, 2023
11c9d34
More snippet tests
czechboy0 Nov 23, 2023
1fbb97e
Support fragment
czechboy0 Nov 23, 2023
7d8625c
Support additionalProperties: true
czechboy0 Nov 23, 2023
8d85996
Support additionalProperties: false
czechboy0 Nov 23, 2023
845a6c5
WIP
czechboy0 Nov 23, 2023
67bf39b
Added support for referenced top multipart schemas
czechboy0 Nov 23, 2023
17340f7
Add a test for referenced schema + encoding
czechboy0 Nov 23, 2023
ce6e234
wip
czechboy0 Nov 23, 2023
bd81c27
Additional properties with schemas work
czechboy0 Nov 24, 2023
4565d06
WIP on documenting
czechboy0 Nov 24, 2023
47d7a02
wip
czechboy0 Nov 24, 2023
e69e607
Remove proposal from this PR
czechboy0 Nov 24, 2023
29b6938
wip
czechboy0 Nov 24, 2023
1bf5030
More docs
czechboy0 Nov 24, 2023
2cf268c
More formatting fixes
czechboy0 Nov 24, 2023
f1505d8
wip
czechboy0 Nov 24, 2023
7a71b13
wip
czechboy0 Nov 24, 2023
5b7e545
Clean soundness
czechboy0 Nov 24, 2023
307fde7
Point runtime to main
czechboy0 Nov 24, 2023
0521c67
Merge branch 'main' into hd-multipart
czechboy0 Nov 24, 2023
502a100
Fix CI
czechboy0 Nov 24, 2023
4a802e6
Clean up complicated conditions
czechboy0 Nov 24, 2023
59024db
PR feedback: added more unit tests
czechboy0 Nov 24, 2023
0e3ad45
Update compat suite
czechboy0 Nov 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ let package = Package(
// Tests-only: Runtime library linked by generated code, and also
// helps keep the runtime library new enough to work with the generated
// code.
.package(url: "https://github.com/apple/swift-openapi-runtime", .upToNextMinor(from: "0.3.6")),
.package(url: "https://github.com/apple/swift-openapi-runtime", branch: "main"),
czechboy0 marked this conversation as resolved.
Show resolved Hide resolved

// Build and preview docs
.package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
Expand Down
70 changes: 70 additions & 0 deletions Sources/PetstoreConsumerTestCore/Assertions.swift
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,73 @@ public func XCTAssertEqualStringifiedData(
if let body = try expression1() { data = try await Data(collecting: body, upTo: .max) } else { data = .init() }
XCTAssertEqualStringifiedData(data, try expression2(), message(), file: file, line: line)
}
fileprivate extension UInt8 {
var asHex: String {
let original: String
switch self {
case 0x0d: original = "CR"
case 0x0a: original = "LF"
default: original = "\(UnicodeScalar(self)) "
}
return String(format: "%02x \(original)", self)
}
}
/// Asserts that the data matches the expected value.
public func XCTAssertEqualData<C1: Collection, C2: Collection>(
_ expression1: @autoclosure () throws -> C1?,
_ expression2: @autoclosure () throws -> C2,
_ message: @autoclosure () -> String = "Data doesn't match.",
file: StaticString = #filePath,
line: UInt = #line
) where C1.Element == UInt8, C2.Element == UInt8 {
do {
guard let actualBytes = try expression1() else {
XCTFail("First value is nil", file: file, line: line)
return
}
let expectedBytes = try expression2()
if ArraySlice(actualBytes) == ArraySlice(expectedBytes) { return }
let actualCount = actualBytes.count
let expectedCount = expectedBytes.count
let minCount = min(actualCount, expectedCount)
print("Printing both byte sequences, first is the actual value and second is the expected one.")
for (index, byte) in zip(actualBytes.prefix(minCount), expectedBytes.prefix(minCount)).enumerated() {
print("\(String(format: "%04d", index)): \(byte.0 != byte.1 ? "x" : " ") \(byte.0.asHex) | \(byte.1.asHex)")
}
let direction: String
let extraBytes: ArraySlice<UInt8>
if actualCount > expectedCount {
direction = "Actual bytes has extra bytes"
extraBytes = ArraySlice(actualBytes.dropFirst(minCount))
} else if expectedCount > actualCount {
direction = "Actual bytes is missing expected bytes"
extraBytes = ArraySlice(expectedBytes.dropFirst(minCount))
} else {
direction = ""
extraBytes = []
}
if !extraBytes.isEmpty {
print("\(direction):")
for (index, byte) in extraBytes.enumerated() {
print("\(String(format: "%04d", minCount + index)): \(byte.asHex)")
}
}
XCTFail(
"Actual stringified data '\(String(decoding: actualBytes, as: UTF8.self))' doesn't equal to expected stringified data '\(String(decoding: expectedBytes, as: UTF8.self))'. Details: \(message())",
file: file,
line: line
)
} catch { XCTFail(error.localizedDescription, file: file, line: line) }
}
/// Asserts that the data matches the expected value.
public func XCTAssertEqualData<C: Collection>(
_ expression1: @autoclosure () throws -> HTTPBody?,
_ expression2: @autoclosure () throws -> C,
_ message: @autoclosure () -> String = "Data doesn't match.",
file: StaticString = #filePath,
line: UInt = #line
) async throws where C.Element == UInt8 {
let data: Data
if let body = try expression1() { data = try await Data(collecting: body, upTo: .max) } else { data = .init() }
XCTAssertEqualData(data, try expression2(), message(), file: file, line: line)
}
97 changes: 97 additions & 0 deletions Sources/PetstoreConsumerTestCore/Common.swift
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,103 @@ public extension Data {
static var quotedEfghString: String { #""efgh""# }

static var efgh: Data { Data(efghString.utf8) }

static let crlf: ArraySlice<UInt8> = [0xd, 0xa]

static var multipartBodyString: String { String(decoding: multipartBodyAsSlice, as: UTF8.self) }

static var multipartBodyAsSlice: [UInt8] {
var bytes: [UInt8] = []
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-disposition: form-data; name="efficiency""#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-length: 3"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "4.2".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-disposition: form-data; name="name""#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-length: 21"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "Vitamin C and friends".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__--".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
return bytes
}

static var multipartBody: Data { Data(multipartBodyAsSlice) }

static var multipartTypedBodyAsSlice: [UInt8] {
var bytes: [UInt8] = []
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-disposition: form-data; filename="process.log"; name="log""#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-length: 35"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-type: text/plain"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"x-log-type: unstructured"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "here be logs!\nand more lines\nwheee\n".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-disposition: form-data; filename="fun.stuff"; name="keyword""#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-length: 3"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-type: text/plain"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "fun".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)

bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-disposition: form-data; filename="barfoo.txt"; name="foobar""#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-length: 0"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-disposition: form-data; name="metadata""#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-length: 42"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-type: application/json; charset=utf-8"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "{\n \"createdAt\" : \"2023-01-18T10:04:11Z\"\n}".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-disposition: form-data; name="keyword""#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-length: 3"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: #"content-type: text/plain"#.utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "joy".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: "--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__".utf8)
bytes.append(contentsOf: "--".utf8)
bytes.append(contentsOf: crlf)
bytes.append(contentsOf: crlf)
return bytes
}
}

public extension HTTPRequest {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ struct VariableDescription: Equatable, Codable {
/// The name of the variable.
///
/// For example, in `let foo = 42`, `left` is `foo`.
var left: String
var left: Expression

/// The type of the variable.
///
Expand Down Expand Up @@ -1106,6 +1106,49 @@ extension Declaration {
setter: [CodeBlock]? = nil,
modify: [CodeBlock]? = nil

) -> Self {
.variable(
accessModifier: accessModifier,
isStatic: isStatic,
kind: kind,
left: .identifierPattern(left),
type: type,
right: right,
getter: getter,
getterEffects: getterEffects,
setter: setter,
modify: modify
)
}

/// A variable declaration.
///
/// For example: `let foo = 42`.
/// - Parameters:
/// - accessModifier: An access modifier.
/// - isStatic: A Boolean value that indicates whether the variable
/// is static.
/// - kind: The variable binding kind.
/// - left: The name of the variable.
/// - type: The type of the variable.
/// - right: The expression to be assigned to the variable.
/// - getter: Body code for the getter of the variable.
/// - getterEffects: Effects of the getter.
/// - setter: Body code for the setter of the variable.
/// - modify: Body code for the `_modify` accessor.
/// - Returns: Variable declaration.
static func variable(
accessModifier: AccessModifier? = nil,
isStatic: Bool = false,
kind: BindingKind,
left: Expression,
type: ExistingTypeDescription? = nil,
right: Expression? = nil,
getter: [CodeBlock]? = nil,
getterEffects: [FunctionKeyword] = [],
setter: [CodeBlock]? = nil,
modify: [CodeBlock]? = nil

) -> Self {
.variable(
.init(
Expand Down Expand Up @@ -1521,14 +1564,6 @@ extension MemberAccessDescription {
static func dot(_ member: String) -> Self { .init(right: member) }
}

extension Expression: ExpressibleByStringLiteral, ExpressibleByNilLiteral, ExpressibleByArrayLiteral {
init(arrayLiteral elements: Expression...) { self = .literal(.array(elements)) }

init(stringLiteral value: String) { self = .literal(.string(value)) }

init(nilLiteral: ()) { self = .literal(.nil) }
}

extension LiteralDescription: ExpressibleByStringLiteral, ExpressibleByNilLiteral, ExpressibleByArrayLiteral {
init(arrayLiteral elements: Expression...) { self = .array(elements) }

Expand All @@ -1544,14 +1579,14 @@ extension VariableDescription {
/// For example `var foo = 42`.
/// - Parameter name: The name of the variable.
/// - Returns: A new mutable variable declaration.
static func `var`(_ name: String) -> Self { Self.init(kind: .var, left: name) }
static func `var`(_ name: String) -> Self { Self.init(kind: .var, left: .identifierPattern(name)) }

/// Returns a new immutable variable declaration.
///
/// For example `let foo = 42`.
/// - Parameter name: The name of the variable.
/// - Returns: A new immutable variable declaration.
static func `let`(_ name: String) -> Self { Self.init(kind: .let, left: name) }
static func `let`(_ name: String) -> Self { Self.init(kind: .let, left: .identifierPattern(name)) }
}

extension Expression {
Expand All @@ -1563,10 +1598,6 @@ extension Expression {
func equals(_ rhs: Expression) -> AssignmentDescription { .init(left: self, right: rhs) }
}

extension FunctionArgumentDescription: ExpressibleByStringLiteral {
init(stringLiteral value: String) { self = .init(expression: .literal(.string(value))) }
}

extension FunctionSignatureDescription {
/// Returns a new function signature description that has the access
/// modifier updated to the specified one.
Expand Down
35 changes: 25 additions & 10 deletions Sources/_OpenAPIGeneratorCore/Renderer/TextBasedRenderer.swift
Original file line number Diff line number Diff line change
Expand Up @@ -543,18 +543,21 @@ struct TextBasedRenderer: RendererProtocol {
/// Renders the specified variable declaration.
func renderVariable(_ variable: VariableDescription) {
do {
var words: [String] = []
if let accessModifier = variable.accessModifier { words.append(renderedAccessModifier(accessModifier)) }
if variable.isStatic { words.append("static") }
words.append(renderedBindingKind(variable.kind))
let labelWithOptionalType: String
if let accessModifier = variable.accessModifier {
writer.writeLine(renderedAccessModifier(accessModifier) + " ")
writer.nextLineAppendsToLastLine()
}
if variable.isStatic {
writer.writeLine("static ")
writer.nextLineAppendsToLastLine()
}
writer.writeLine(renderedBindingKind(variable.kind) + " ")
writer.nextLineAppendsToLastLine()
renderExpression(variable.left)
if let type = variable.type {
labelWithOptionalType = "\(variable.left): \(renderedExistingTypeDescription(type))"
} else {
labelWithOptionalType = variable.left
writer.nextLineAppendsToLastLine()
writer.writeLine(": \(renderedExistingTypeDescription(type))")
}
words.append(labelWithOptionalType)
writer.writeLine(words.joinedWords())
}

if let right = variable.right {
Expand Down Expand Up @@ -883,3 +886,15 @@ fileprivate extension String {
/// - Returns: A new string where each line has been transformed using the given closure.
func transformingLines(_ work: (String) -> String) -> [String] { asLines().map(work) }
}

extension TextBasedRenderer {

/// Returns the provided expression rendered as a string.
/// - Parameter expression: The expression.
/// - Returns: The string representation of the expression.
static func renderedExpressionAsString(_ expression: Expression) -> String {
let renderer = TextBasedRenderer.default
renderer.renderExpression(expression)
return renderer.renderedContents()
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ extension ClientFileTranslator {
)
requestBlocks.append(.expression(requestBodyExpr))
} else {
requestBodyReturnExpr = nil
requestBodyReturnExpr = .literal(nil)
}

let returnRequestExpr: Expression = .return(.tuple([.identifierPattern("request"), requestBodyReturnExpr]))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ enum AllOrAnyOf {
case anyOf
}

extension FileTranslator {
extension TypesFileTranslator {

/// Returns a declaration for an allOf or anyOf schema.
///
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
//===----------------------------------------------------------------------===//
import OpenAPIKit

extension FileTranslator {
extension TypesFileTranslator {

/// Returns a list of declarations for an array schema.
///
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ extension FileTranslator {
trailingCodeBlocks: [
.expression(
.assignment(
left: .identifierPattern("additionalProperties"),
left: .identifierPattern(Constants.AdditionalProperties.variableName),
right: .try(
.identifierPattern("decoder").dot("decodeAdditionalProperties")
.call([knownKeysFunctionArg])
Expand Down Expand Up @@ -86,7 +86,12 @@ extension FileTranslator {
.expression(
.try(
.identifierPattern("encoder").dot("encodeAdditionalProperties")
.call([.init(label: nil, expression: .identifierPattern("additionalProperties"))])
.call([
.init(
label: nil,
expression: .identifierPattern(Constants.AdditionalProperties.variableName)
)
])
)
)
]
Expand Down
Loading