Outcomes are 'None' or 'Inapplicable' for default rules

[drc-nloftsgard]

I wrote a script (see https://gist.github.com/drc-nloftsgard/588ddfef8d7fdd42cb7df4a50ecda884) to scrape a page and am not getting the outcomes that I expect.

The command to run this script is:
cross-env ./node_modules/.bin/ts-node ./src/app/reports/accessibility/alfa.tests.ts

Am I missing something in the setup of this test? Is there a certain TypeScript version or dependencies required that would fix this? What else can I try to troubleshoot the issue?

[kasperisager]

Thanks for reaching out! The issue you're seeing is related to the use of iterables. Some iterables, such as the one containing the outcomes, can only be iterated once while your code attempts to iterate it several times. On line 50, the iterable of outcomes is fully consumed to determine its size. Then, on line 57, the code attempts to iterate the iterable again once for every outcome, but at this point the iterable is actually empty. That's why you're seeing None returned every time the code attempts to read an outcome.

The easiest solution would be to instead use a for ... of loop to ensure that the iterable is only iterated once:

for (const outcome of outcomes) {
  console.log(outcome);
}

The behaviour you're seeing on line 8 is also a little concerning so I'll look into that.

[kasperisager]

I'm suspecting that you've got two incompatible versions of @siteimprove/alfa-web installed, which perhaps isn't surprising as none of the currently published versions of that package are compatible. The wonders of pre-releases!

Could you send me the output of yarn why @siteimprove/alfa-web or npm ls @siteimprove/alfa-web?

[drc-nloftsgard]

npm ls @siteimprove/alfa-web yields

│ ├─┬ @siteimprove/alfa-chai@0.0.1
│ │ └── @siteimprove/alfa-web@0.0.1  deduped
│ └── @siteimprove/alfa-web@0.0.1  deduped
├─┬ @siteimprove/alfa-jasmine@0.0.1
│ └─┬ @siteimprove/alfa-assert@0.0.1
│   ├─┬ @siteimprove/alfa-rules@0.0.1
│   │ └── @siteimprove/alfa-web@0.0.1  deduped
│   └── @siteimprove/alfa-web@0.0.1  deduped
├─┬ @siteimprove/alfa-rules@0.1.0
│ └── @siteimprove/alfa-web@0.1.0 
├─┬ @siteimprove/alfa-scraper@0.1.0
│ ├─┬ @siteimprove/alfa-puppeteer@0.1.0
│ │ ├─┬ @siteimprove/alfa-chai@0.1.0
│ │ │ ├─┬ @siteimprove/alfa-assert@0.1.0
│ │ │ │ ├─┬ @siteimprove/alfa-rules@0.1.0
│ │ │ │ │ └── @siteimprove/alfa-web@0.1.0  deduped
│ │ │ │ └── @siteimprove/alfa-web@0.1.0  deduped
│ │ │ └── @siteimprove/alfa-web@0.1.0  deduped
│ │ └── @siteimprove/alfa-web@0.1.0 
│ └── @siteimprove/alfa-web@0.1.0 
└── @siteimprove/alfa-web@0.0.1 

[drc-nloftsgard]

Thanks for the quick response on the main issue. I took out the code to determine size and updated to use for ... of over the outcomes (gist updated). However, all of the outcomes show as 'inapplicable' (example below of outcome.toJSON()). Anything else that I might be missing or doing incorrectly?

{ 
  outcome: 'inapplicable',
  rule:
   {
     type: 'atomic',
     uri: 'https://siteimprove.githu.io/sanshikan/rules/sia-r24.html' 
   }
}

[kasperisager]

With every @siteimprove/alfa-* package at v0.1.0 I'm seeing plenty of applicable outcomes. Could you try upgrading the packages and run the code again?

[drc-nloftsgard]

Sure, I set them as below (devDependencies in my package.json) and ran npm install

    "@siteimprove/alfa-act": "0.1.0",
    "@siteimprove/alfa-angular": "0.1.0",
    "@siteimprove/alfa-device": "0.1.0",
    "@siteimprove/alfa-dom": "0.1.0",
    "@siteimprove/alfa-jasmine": "0.1.0",
    "@siteimprove/alfa-rules": "0.1.0",
    "@siteimprove/alfa-scraper": "0.1.0",
    "@siteimprove/alfa-web": "0.1.0",

I get the same 'inapplicable' for all rules. I'm experimented with different values/settings in my tsconfig.json but that doesn't seem to help or change the results.

Is there any other debugging I can do? What other output would help figure this out?

[kasperisager]

The dependencies look correct so that's good 👍 Could you try filtering out inapplicable outcomes to see if every rule is indeed inapplicable? One way would be:

[...outcomes].filter(Outcome.isApplicable);

[drc-nloftsgard]

Indeed, I verified all rules are inapplicable.

[kasperisager]

That's really odd and not at all what I'm seeing on my end 🤔 Does the screenshot look correct?

[kasperisager]

Here's what I'm seeing as a result of [...outcomes].map(outcome => outcome.toEARL()): result.zip

[drc-nloftsgard]

The screenshot looks correct. I created a new project/repository, alfa-test, and I get the expected Outcomes!

So there must be something in my other (private company) repository that's causing this issue.

[kasperisager]

Wonderful! 👏 Do let me know if anything else pops up!