AI-screened eye pics diagnose childhood autism with 100% accuracy::undefined

  • jet@hackertalks.com
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    2
    ·
    11 months ago

    Then somebody’s lying with creative application of 100% accuracy rates.

    The confidence interval of the sequence you describe is not 100%

    • eggymachus@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      10
      ·
      11 months ago

      From TFA:

      For ASD screening on the test set of images, the AI could pick out the children with an ASD diagnosis with a mean area under the receiver operating characteristic (AUROC) curve of 1.00. AUROC ranges in value from 0 to 1. A model whose predictions are 100% wrong has an AUROC of 0.0; one whose predictions are 100% correct has an AUROC of 1.0, indicating that the AI’s predictions in the current study were 100% correct. There was no notable decrease in the mean AUROC, even when 95% of the least important areas of the image – those not including the optic disc – were removed.

      They at least define how they get the 100% value, but I’m not an AIologist so I can’t tell if it is reasonable.