Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update gitpod.yaml to build multiple packages #25245

Merged
merged 6 commits into from
Apr 25, 2023
Merged

Conversation

pahud
Copy link
Contributor

@pahud pahud commented Apr 21, 2023

With #25224 , gitpod now prebuilds the aws-cdk-lib for us and is potentially saving 7-9 minutes with that. This PR makes gitpod continue to build @aws-cdk-testing/framework-integ so developers don't have to spend their time waiting for the all integ building.

Closes #.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented Apr 21, 2023

@aws-cdk-automation aws-cdk-automation requested a review from a team April 21, 2023 21:13
@github-actions github-actions bot added the p2 label Apr 21, 2023
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Apr 21, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@pahud pahud changed the title gitpod: update gitpod.yaml to build multiple packages chore: update gitpod.yaml to build multiple packages Apr 21, 2023
@aws-cdk-automation aws-cdk-automation dismissed their stale review April 21, 2023 21:22

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@pahud pahud marked this pull request as ready for review April 22, 2023 02:36
@pahud pahud marked this pull request as draft April 22, 2023 02:39
@pahud
Copy link
Contributor Author

pahud commented Apr 22, 2023

The prebuild from this PR is saving 16min now.

image

@pahud pahud marked this pull request as ready for review April 22, 2023 13:10
.gitpod.yml Outdated
Comment on lines 14 to 17
# build aws-cdk-lib
cd packages/aws-cdk-lib && ../../scripts/buildup
# build framework-integ
cd ../@aws-cdk-testing/framework-integ && ../../../scripts/buildup
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like we'll end up building some packages twice (e.g. aws-cdk-lib, some of the build tools).

Could you refactor this to using lerna directly to build the two target packages instead of using buildup? That way lerna should be able to figure out exactly which packages to build and will only build them once.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrgrain Thank you. I actually encountered some errors using lerna in gitpod. I will try again later today and report back.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mrgrain

When I run npx lerna run build --scope=aws-cdk-lib at aws-cdk-lib in gitpod I got this error message

  aws-ecr-assets/lib/image-asset.ts:56:14
    56 export class Platform {
                    ~~~~~~~~
    The declaring class is introduced here
aws-lambda/lib/lambda-version.ts:180:19 - warning JSII5019: The property name "version" conflicts with the declaring class "Version". This will result in renaming the class to "_Version" in C#. Consider renaming "version".
180   public readonly version: string;
                      ~~~~~~~
  aws-lambda/lib/lambda-version.ts:114:14
    114 export class Version extends QualifiedFunctionBase implements IVersion {
                     ~~~~~~~
    The declaring class is introduced here
Build times for aws-cdk-lib: Total time (4m36.1s) | /workspace/aws-cdk/tools/@aws-cdk/cdk-build-tools/node_modules/jsii/bin/jsii (4m3.5s) | ts-node ./scripts/verify-imports-resolve-same.ts && ts-node ./scripts/verify-imports-shielded.ts && ts-node ./cx-api/build-tools/flag-report.ts (20.8s) | ts-node -P tsconfig.dev.json scripts/gen.ts (7.2s) | npx ts-node -P tsconfig.dev.json region-info/build-tools/generate-static-data.ts && node aws-events-targets/build-tools/gen.js && (cp -f $(node -p 'require.resolve("aws-sdk/apis/metadata.json")') custom-resources/lib/aws-custom-resource/sdk-api-metadata.json && rm -rf custom-resources/test/aws-custom-resource/cdk.out) && (rm -rf core/test/fs/fixtures && cd core/test/fs && tar -xzf fixtures.tar.gz) && (rm -rf assets/test/fs/fixtures && cd assets/test/fs && tar -xzvf fixtures.tar.gz) (3.0s)
node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: read ENOTCONN
    at tryReadStart (node:net:662:20)
    at Socket._read (node:net:673:5)
    at Socket.Readable.read (node:internal/streams/readable:487:10)
    at Socket.read (node:net:729:39)
    at new Socket (node:net:449:12)
    at Object.Socket (node:net:318:41)
    at createSocket (node:internal/child_process:335:14)
    at ChildProcess.spawn (node:internal/child_process:452:23)
    at spawn (node:child_process:733:9)
    at execFile (node:child_process:331:17)
    at /workspace/aws-cdk/node_modules/nx/src/tasks-runner/cache.js:151:46
    at new Promise (<anonymous>)
    at Cache.<anonymous> (/workspace/aws-cdk/node_modules/nx/src/tasks-runner/cache.js:150:20)
    at Generator.next (<anonymous>)
    at /workspace/aws-cdk/node_modules/tslib/tslib.js:167:75
    at new Promise (<anonymous>)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -107,
  code: 'ENOTCONN',
  syscall: 'read'
}

That's why I switched to buildup. Any thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, it appears the cache is causing an issue. Not sure why exactly, maybe because the artifacts are so large?!
But this works for me:

npx lerna run build --scope=aws-cdk-lib --scope=@aws-cdk-testing/framework-integ --skip-nx-cache

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I've learned something new today 👍

Will wait for the prebuild and see how it works!

.gitpod.yml Outdated Show resolved Hide resolved
Co-authored-by: Momo Kornher <mail@moritzkornher.de>
@pahud
Copy link
Contributor Author

pahud commented Apr 25, 2023

It works! Potentially saving 21minutes!

image

@mergify
Copy link
Contributor

mergify bot commented Apr 25, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 69c9c5f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 9e0a8d7 into aws:main Apr 25, 2023
@mergify
Copy link
Contributor

mergify bot commented Apr 25, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

madeline-k pushed a commit that referenced this pull request Apr 27, 2023
With #25224 , gitpod now prebuilds the `aws-cdk-lib` for us and is potentially saving 7-9 minutes with that. This PR makes gitpod continue to build `@aws-cdk-testing/framework-integ` so developers don't have to spend their time waiting for the all integ building. 

Closes #<issue number here>.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants