Brenner Cruvinel - compressãoAI researcher and product designer building at the edge of computer science and mental health.Zola2026-03-12T00:00:00+00:00https://brennercruvinel.blog/tags/compressao/atom.xmlCompressão de Imagem para Melhorar Computer Vision2026-03-12T00:00:00+00:002026-03-12T00:00:00+00:00
Brenner Cruvinel
https://brennercruvinel.blog/blog/compressao-imagem-computer-vision/<h2 id="using-image-compression-to-improve-computer-vision-part-1">using image compression to improve computer vision, part 1</h2>
<p>clipping de referencia. Kaleigh Mentzer, Granica, 4 de novembro de 2023, fonte https://www.granica.ai/blog/using-image-compression-to-improve-computer-vision-part-1 . a tese que importa pro dermML: imagem de treino comprime ate niveis altos sem perda de acuracia, porque o modelo ignora a informacao de alta frequencia que a compressao joga fora. a parte 2 esta em [[image-compression-improve-cv-part-2]].</p>
<h3 id="a-serie">a serie</h3>
<p>multi-part series sobre como time de ML pode usar compressao com perda pra melhorar modelo de computer vision. a parte 1 mostra que imagem de treino pode ser comprimida pra reduzir storage sem sacrificar performance. a parte 2 mostra como o espaco liberado acomoda dado de treino adicional comprimido, melhorando performance dentro de orcamento fixo.</p>
<h3 id="o-problema-de-storage">o problema de storage</h3>
<p>modelo de computer vision state-of-the-art demanda muito dado de imagem, o que cria requisito de storage caro. muito time de ML enfrenta restricao implicita de storage que resulta em ate 70% do dado novo sendo arquivado ou deletado.</p>
<h3 id="fundamentos-de-compressao">fundamentos de compressao</h3>
<p>lossless vs lossy: a lossless permite reconstrucao exata mas tipicamente alcanca so ~50% de reducao pra imagem fotografica. a lossy permite degradacao perceptual sem afetar a performance do modelo, com potencial de comprimir a imagem ate poucos pontos percentuais do tamanho original.</p>
<p>distancia de Butteraugli: a metrica de distancia de Butteraugli mede a mudanca de qualidade percebida. JPEG XL e codec moderno similar permitem comprimir ate uma distancia de Butteraugli especificada minimizando o tamanho do arquivo.</p>
<p>data compression rate (DCR): DCR = 1 - (compressed size / uncompressed size). DCR maior indica arquivo menor. o teste mostra que o DCR cresce rapido conforme a distancia de Butteraugli aumenta.</p>
<h3 id="resultados-experimentais">resultados experimentais</h3>
<h4 id="classificacao-de-imagem">classificacao de imagem</h4>
<p>dataset Food101 (75.750 train, 25.250 test, 101 categorias de comida). modelo testado: Vision Transformer (ViT) pre-treinado em ImageNet, SwAV features com ResNet-50. achado: a compressao tem impacto minimo na acuracia ate exceder ~70% de taxa de compressao. o modelo e mais sensivel a compressao do test set do que a do dado de treino.</p>
<h4 id="object-detection">object detection</h4>
<p>dataset PASCAL VOC 2012 (5.717 imagens de treino, 20 classes de objeto). modelo Faster R-CNN com backbone ResNet-50. metrica mean average precision (mAP). resultado: o nivel de compressao do test set determina a acuracia, a compressao do training set tem impacto minimo.</p>
<h4 id="semantic-segmentation">semantic segmentation</h4>
<p>dataset Cityscapes (5.000 imagens com anotacao semantica). modelo Segformer com encoder ImageNet pre-treinado. metrica mean intersection over union (mIoU) e mean accuracy (mAcc). achado-chave: modelo pode ser treinado em dado comprimido em 97%, requerendo so 3% do storage original de treino, sem degradacao significativa nessa metrica.</p>
<h3 id="por-que-a-compressao-tem-impacto-limitado">por que a compressao tem impacto limitado</h3>
<p>a compressao com perda descarta a informacao perceptualmente menos importante primeiro, tipicamente cor e componente de alta frequencia. o modelo de ML talvez tambem nao dependa muito dessa informacao.</p>
<p>arquitetura JPEG/JPEG XL, os dois formatos: 1. convertem RGB pra YCbCr, 2. downsample do canal de chrominancia (humano e ~3x menos sensivel a cor que a luminancia), 3. dividem a imagem em bloco de 8x8 pixels, 4. aplicam discrete cosine transform (DCT) pra decompor o componente de frequencia, 5. quantizam agressivamente o coeficiente de alta frequencia preservando a precisao do de baixa frequencia.</p>
<p>analise de sensibilidade do modelo: teste com perturbacao de um unico coeficiente DCT em 32 imagens do Food101 mostrou que o modelo e significativamente menos sensivel a mudanca de alta frequencia. isso e evidencia de que o artefato de compressao afeta minimamente a predicao porque o modelo nao depende da informacao de alta frequencia descartada.</p>
<h3 id="conclusao">conclusao</h3>
<p>imagem de treino pode ser comprimida substancialmente sem perda de acuracia relevante em varias tarefas de computer vision. o modelo demonstra robustez ao artefato de compressao, provavelmente porque ignora a informacao de alta frequencia que a compressao remove. tarefa diferente pode exigir nivel de compressao diferente conforme o requisito de acuracia, mas comprimir alem de PNG/JPEG oferece economia de storage significativa com impacto minimo na performance.</p>
<h2 id="using-image-compression-to-improve-computer-vision-part-2">using image compression to improve computer vision, part 2</h2>
<p>clipping de referencia. Granica, fonte https://www.granica.ai/blog/using-image-compression-to-improve-computer-vision-part-2 . continuacao de [[image-compression-improve-cv-part-1]]. o resultado que importa: mais imagem mesmo que mais comprimida melhora o modelo em mais de 10 pontos de acuracia, sem aumentar o custo de storage, quando o modelo opera em regime de dado escasso.</p>
<h3 id="a-oportunidade">a oportunidade</h3>
<p>empresa costuma operar com orcamento fixo de cloud storage pro dado de treino. enquanto a operacao gera dado novo em volume, a restricao de orcamento forca a deletar dataset potencialmente valioso. a compressao de imagem permite reter mais dado de treino dentro do mesmo storage: quanto maior o nivel de compressao, mais imagem adicional cabe.</p>
<h3 id="o-experimento">o experimento</h3>
<p>usa o dataset Food101, 101 tipos de comida pra classificacao. testaram seis niveis de compressao JPEG XL, medindo a compressao por distancia de Butteraugli (0, 2, 4, 6, 8 e 10), onde distancia maior indica mais perda de qualidade.</p>
<p>com orcamento de 0.5 GB de storage:</p>
<ul>
<li>formato JPEG original: 9.985 imagens cabem no orcamento</li>
<li>na distancia de Butteraugli 10: aproximadamente 5.6 vezes mais imagem cabe</li>
</ul>
<p>a analise mostra que mais compressao permite caber substancialmente mais amostra de treino dentro da mesma restricao de storage.</p>
<h3 id="o-resultado">o resultado</h3>
<p>testando um Vision Transformer pre-treinado em ImageNet-21k e fine-tuned em subconjuntos do Food101, ter mais imagem, mesmo que mais comprimida com perda, melhora a performance do modelo.</p>
<p>achado-chave: a acuracia do modelo melhorou mais de 10 pontos percentuais sem aumentar o custo de storage. pra esse modelo operando em regime de dado limitado, com menos de 100 imagens por classe, a quantidade provou ser mais valiosa que a qualidade da imagem.</p>
<h3 id="implicacoes">implicacoes</h3>
<p>conforme o modelo de ML cresce e demanda mais dado de treino pra performance otima, a compressao com perda apresenta uma solucao pratica. o time pode expandir o dataset de treino e potencialmente fazer deploy de modelo mais sofisticado sem cloud storage adicional ou aumento de orcamento.</p>
<p>a analise enfatiza que modelo diferente exibe padrao de scaling diferente. o sucesso depende de entender se um modelo especifico opera numa situacao de dado escasso onde amostra adicional gera melhora relevante.</p>
<h2 id="datasets-de-dermatologia-recuperacao-e-download">datasets de dermatologia, recuperacao e download</h2>
<p>guia de download pra recriar o ambiente completo. o sistema ja suporta busca por similaridade mas os datasets foram perdidos (SSD NVMe). esta nota lista os principais datasets com comando de download e prioridade. o inventario completo do mundo esta em [[dermML-master-dataset-catalog]].</p>
<h3 id="ja-indexados-arquivo-perdido">ja indexados (arquivo perdido)</h3>
<p>HAM10000, Harvard Dataverse. indexado, arquivo perdido. 10.015 imagens (~5GB), dermoscopia, 7 tipos de lesao pigmentada.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> oficial: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/ham10000</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">wget</span><span style="color: light-dark(#2AA198, #2AA198);"> https://isic-challenge-data.s3.amazonaws.com/2018/ISIC2018_Task3_Training_Input.zip</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">unzip</span><span style="color: light-dark(#2AA198, #2AA198);"> ISIC2018_Task3_Training_Input.zip</span></span></code></pre>
<p>diagnostico: melanoma (1113), melanocytic nevus (6705), basal cell carcinoma (514), actinic keratosis (327), benign keratosis (1099), dermatofibroma (115), vascular lesion (142).</p>
<p>BCN20000, Barcelona Dermatology Dataset. indexado, arquivo perdido. 25.331 imagens (~12GB), dermoscopia, 8 categorias.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> requer solicitacao: https://challenge2019.isic-archive.com/</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> ou ISIC Archive: https://www.isic-archive.com/</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/bcn20000</span></span></code></pre>
<p>SCIN, Skin Cancer Image Network. indexado, arquivo perdido. 10.379 imagens (~3GB), fotografia clinica, multiplas condicoes.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> requer registro: https://www.isic-archive.com/</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/scin</span></span></code></pre><h3 id="datasets-a-adicionar">datasets a adicionar</h3>
<p>ISIC 2024 Challenge. 400.000+ imagens, dermoscopia mais clinica. acesso https://challenge2024.isic-archive.com/ .</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/isic2024</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/isic2024</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> download via ISIC API ou Kaggle</span></span></code></pre>
<p>ISIC 2020 Challenge. 33.126 imagens, dermoscopia, melanoma 584 casos.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/isic2020</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/isic2020</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">kaggle</span><span style="color: light-dark(#2AA198, #2AA198);"> competitions</span><span style="color: light-dark(#2AA198, #2AA198);"> download</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">c</span><span style="color: light-dark(#2AA198, #2AA198);"> siim-isic-melanoma-classification</span></span></code></pre>
<p>ISIC 2019 Challenge. 25.331 imagens, dermoscopia, 8 classes.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/isic2019</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/isic2019</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">kaggle</span><span style="color: light-dark(#2AA198, #2AA198);"> competitions</span><span style="color: light-dark(#2AA198, #2AA198);"> download</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">c</span><span style="color: light-dark(#2AA198, #2AA198);"> isic-2019</span></span></code></pre>
<p>ISIC 2018 Challenge. 10.015 imagens (HAM10000), dermoscopia, 7 classes. ja temos como HAM10000.</p>
<p>Fitzpatrick17k. 16.577 imagens (~2GB), fotografia clinica, fototipo I-VI, 114 condicoes.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/fitzpatrick17k</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/fitzpatrick17k</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> GitHub: https://github.com/mattgroh/fitzpatrick17k</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">git</span><span style="color: light-dark(#2AA198, #2AA198);"> clone</span><span style="color: light-dark(#2AA198, #2AA198);"> https://github.com/mattgroh/fitzpatrick17k.git</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> fitzpatrick17k</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">python</span><span style="color: light-dark(#2AA198, #2AA198);"> download_images.py</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> ou baixar direto (se disponivel):</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> wget https://vault.sfu.ca/index.php/s/cMuxZNzk6UUHNmX (arquivo do paper DermSynth3D)</span></span></code></pre>
<p>metadado: CSV com URL, diagnostico, fototipo Fitzpatrick.</p>
<p>PAD-UFES-20. 2.298 imagens (~500MB), fotografia clinica com smartphone, 6 doencas. metadado clinico completo (idade, regiao, historico).</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/pad-ufes-20</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/pad-ufes-20</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">kaggle</span><span style="color: light-dark(#2AA198, #2AA198);"> datasets</span><span style="color: light-dark(#2AA198, #2AA198);"> download</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">d</span><span style="color: light-dark(#2AA198, #2AA198);"> mahdavi1202/skin-cancer</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">unzip</span><span style="color: light-dark(#2AA198, #2AA198);"> skin-cancer.zip</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> ou direto: https://data.mendeley.com/datasets/zr7vgbcyr2/1</span></span></code></pre>
<p>classes: Basal Cell Carcinoma (BCC), Squamous Cell Carcinoma (SCC), Actinic Keratosis (ACK), Seborrheic Keratosis (SEK), Bowen’s Disease (BOD), Melanoma (MEL).</p>
<p>Derm7pt. 2.000+ imagens, dermoscopia, anotacao 7-point checklist pra melanoma.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/derm7pt</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/derm7pt</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> requer registro: http://derm.cs.sfu.ca/</span></span></code></pre>
<p>7-point checklist: 1. atypical pigment network, 2. blue-whitish veil, 3. atypical vascular pattern, 4. irregular streaks, 5. irregular dots/globules, 6. irregular blotches, 7. regression structures.</p>
<p>PH2, Pedro Hispano Hospital. 200 imagens (~50MB), dermoscopia, segmentacao manual por dermatologista.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/ph2</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/ph2</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> oficial: https://www.fc.up.pt/addi/ph2%20database.html</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">wget</span><span style="color: light-dark(#2AA198, #2AA198);"> https://www.dropbox.com/s/k88qukc20ljnbuo/PH2Dataset.rar</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">unrar</span><span style="color: light-dark(#2AA198, #2AA198);"> x</span><span style="color: light-dark(#2AA198, #2AA198);"> PH2Dataset.rar</span></span></code></pre>
<p>classes: common nevus (80), atypical nevus (80), melanoma (40).</p>
<p>DermoFit Image Library. 1.300 imagens, fotografia clinica de alta qualidade, 10 classes.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/dermofit</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/dermofit</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> requer licenca paga: https://licensing.edinburgh-innovations.ed.ac.uk/product/dermofit-image-library</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> ~500 GBP pra licenca academica perpetua</span></span></code></pre>
<p>classes: Actinic Keratosis, Basal Cell Carcinoma, Melanocytic Nevus, Squamous Cell Carcinoma, Seborrhoeic Keratosis, Intraepithelial Carcinoma, Pyogenic Granuloma, Haemangioma, Dermatofibroma, Melanoma.</p>
<p>DDI, Diverse Dermatology Images. 656 imagens, fotografia clinica, foco em tom de pele diverso.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/ddi</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/ddi</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> GitHub: https://github.com/ddrestrepo/diverse-dermatology-images</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">git</span><span style="color: light-dark(#2AA198, #2AA198);"> clone</span><span style="color: light-dark(#2AA198, #2AA198);"> https://github.com/ddrestrepo/diverse-dermatology-images.git</span></span></code></pre>
<p>FUSeg, Foot Ulcer Segmentation. 1.210 imagens, fotografia clinica de ulcera, segmentacao pixel-wise.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/fuseg</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/fuseg</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> GitHub: https://github.com/uwm-bigdata/wound-segmentation</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">git</span><span style="color: light-dark(#2AA198, #2AA198);"> clone</span><span style="color: light-dark(#2AA198, #2AA198);"> https://github.com/uwm-bigdata/wound-segmentation.git</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cp</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">r</span><span style="color: light-dark(#2AA198, #2AA198);"> wound-segmentation/data/Foot</span><span style="color: light-dark(#CB4B16, #CB4B16);">\ </span><span style="color: light-dark(#2AA198, #2AA198);">Ulcer</span><span style="color: light-dark(#CB4B16, #CB4B16);">\ </span><span style="color: light-dark(#2AA198, #2AA198);">Segmentation</span><span style="color: light-dark(#CB4B16, #CB4B16);">\ </span><span style="color: light-dark(#2AA198, #2AA198);">Challenge/</span><span style="color: light-dark(#268BD2, #268BD2);">*</span><span style="color: light-dark(#2AA198, #2AA198);"> .</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> ou mirror: https://vault.sfu.ca/index.php/s/2mb8kZg8wOltptT</span></span></code></pre>
<p>Pratheepan, Skin Detection. 78 imagens, fotografia clinica (face), segmentacao de pele vs background.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/pratheepan</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/pratheepan</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> website: https://web.fsktm.um.edu.my/~cschan/downloads_skin_dataset.html</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> Google Drive link na pagina</span></span></code></pre>
<p>SD-198 e SD-260. 6.584 imagens (198 classes) mais 40.000+ (260 classes), crawled da web, fonte DermNet, Dermis.net, DermQuest.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/sd-{198,260}</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> paper: https://arxiv.org/abs/1911.08716</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> requer contato com autores</span></span></code></pre>
<p>DermNet NZ. 23.000+ imagens, fotografia clinica educacional, 600+ condicoes.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/dermnet</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/dermnet</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> Kaggle (subset):</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">kaggle</span><span style="color: light-dark(#2AA198, #2AA198);"> datasets</span><span style="color: light-dark(#2AA198, #2AA198);"> download</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">d</span><span style="color: light-dark(#2AA198, #2AA198);"> shubhamgoel27/dermnet</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">unzip</span><span style="color: light-dark(#2AA198, #2AA198);"> dermnet.zip</span></span></code></pre><h3 id="datasets-sinteticos">datasets sinteticos</h3>
<p>DermSynth3D. ~52.000 imagens sinteticas (~15GB), renderizacao 3D com anotacao rica (segmentacao, depth, anatomia, bbox). ja documentado em DERMSYNTH3D_SETUP.md.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/dermsynth3d</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/dermsynth3d</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> requer solicitacao: https://cvi2.uni.lu/3dbodytexdermsynth/</span></span></code></pre>
<p>3DBodyTex.v1. ~50GB, scan 3D texturizado de corpo humano, base pro DermSynth3D.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">mkdir</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">p</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/3dbodytex-1.1-highres</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">cd</span><span style="color: light-dark(#2AA198, #2AA198);"> ~/dermML-models/datasets/3dbodytex-1.1-highres</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> requer licenca: https://cvi2.uni.lu/3dbodytexv1/</span></span></code></pre><h3 id="estrutura-de-diretorios">estrutura de diretorios</h3>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#859900, #859900);">~</span><span>/dermML-models/</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> datasets/</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> ham10000/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 10k dermoscopia</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> bcn20000/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 25k dermoscopia</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> scin/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 10k clinica</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> isic2024/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 400k+ (novo)</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> isic2020/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 33k melanoma</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> isic2019/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 25k 8-classes</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> fitzpatrick17k/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 16k diversidade</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> pad-ufes-20/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 2k smartphone</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> derm7pt/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 2k 7-point checklist</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> ph2/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 200 alta qualidade</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> dermofit/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 1.3k licenca paga</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> ddi/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 656 diversidade</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> fuseg/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 1.2k ulcera</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> pratheepan/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 78 skin detection</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> sd-198/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 6k 198 classes</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> sd-260/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 40k 260 classes</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> dermnet/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 23k educacional</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> dermsynth3d/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 52k sintetico</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> 3dbodytex-1.1-highres/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> scan 3D</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> similarity/</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> embeddings.index</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> FAISS atual (HAM+BCN+SCIN)</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> metadata.db</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> SQLite metadado</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> dermsynth/</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> sinteticos separados</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> embeddings_dermsynth.index</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> metadata_dermsynth.db</span></span></code></pre><h3 id="prioridade-de-download">prioridade de download</h3>
<p>tier 1, essencial, ja temos mas perdido: HAM10000 (10k dermoscopia, base), BCN20000 (25k dermoscopia, volume), SCIN (10k clinica, diversidade).</p>
<p>tier 2, muito importante, adicionar: ISIC 2024 (400k+, maior disponivel), Fitzpatrick17k (16k, diversidade de tom), PAD-UFES-20 (2k, smartphone real-world).</p>
<p>tier 3, importante, qualidade: PH2 (200, alta qualidade, baseline), Derm7pt (2k, 7-point checklist clinico), DermNet (23k, educacional, muitas classes).</p>
<p>tier 4, especializado: DermSynth3D (52k sintetico, anotacao rica), FUSeg (1.2k ulcera, nicho mas util), DDI (656, benchmark de diversidade).</p>
<p>tier 5, opcional/caro: DermoFit (1.3k, licenca 500 GBP), SD-198/260 (46k, requer contato).</p>
<h3 id="tamanho-total-estimado">tamanho total estimado</h3>
<table><thead><tr><th>tier</th><th>datasets</th><th>imagens</th><th>tamanho aprox</th></tr></thead><tbody>
<tr><td>1</td><td>HAM+BCN+SCIN</td><td>45k</td><td>~20GB</td></tr>
<tr><td>2</td><td>ISIC24+Fitz+PAD</td><td>418k</td><td>~50GB</td></tr>
<tr><td>3</td><td>PH2+Derm7pt+DermNet</td><td>27k</td><td>~15GB</td></tr>
<tr><td>4</td><td>DermSynth+FUSeg+DDI</td><td>54k</td><td>~20GB</td></tr>
<tr><td>5</td><td>DermoFit+SD</td><td>48k</td><td>~25GB</td></tr>
<tr><td>total</td><td>all</td><td>~592k</td><td>~130GB</td></tr>
</tbody></table>
<h3 id="proximos-passos">proximos passos</h3>
<ol>
<li>estrutura de diretorios criada</li>
<li>criar script <code>scripts/download_all_datasets.sh</code></li>
<li>atualizar <code>compute_embeddings.py</code> pra multiplos datasets</li>
<li>reindexar FAISS com todos os datasets</li>
<li>atualizar metadata.db com campo novo</li>
<li>testar busca com 592k imagens</li>
</ol>
<h3 id="gaps-por-escala-de-impacto-maior-primeiro">gaps por escala de impacto, maior primeiro</h3>
<ol>
<li>diversidade de tons de pele Fitzpatrick IV-VI em escala (>50k imagens). afeta 2.5B+ pessoas sem representacao algoritmica adequada.</li>
<li>dermatologia para tons asiaticos (leste mais sul da Asia). 4B+ pessoas, modelo atual nao validado pra essa semiotica dermoscopica.</li>
<li>WSI dermatologico aberto em escala (>10k laminas 40x). a patologia digital inteira depende de dado que nao existe em formato publico.</li>
<li>longitudinal, lesao -> evolucao -> outcome ao longo do tempo. elimina biopsia desnecessaria globalmente, milhoes por ano.</li>
<li>multimodal, dermoscopia mais histopatologia mais genomica linkados por paciente. precision dermatology nao existe sem isso.</li>
<li>dataset EHR multi-center fora do eixo EUA/Europa. MIMIC e single-center Boston, eICU e Philips-only, 85% do planeta nao esta representado.</li>
<li>dermatologia mais saude mental (comorbidade linkada, DLQI mais PHQ-9 mais imagem). psoriase, eczema, acne tem impacto psiquiatrico massivo nao quantificado em escala.</li>
<li>dataset padronizado de estetica antes/depois com outcome measure validada. mercado de 100B+ USD sem dado cientifico de qualidade.</li>
<li>NLP em portugues clinico dermatologico. 80k+ dermatologistas no Brasil documentando em texto livre sem nenhum modelo treinado pra esse dominio.</li>
<li>dado de procedimento estetico nao-cirurgico (filler, toxina botulinica, laser) com follow-up. segmento de maior crescimento global, zero dataset.</li>
<li>dermatologia pediatrica em escala. crianca quase ausente dos datasets existentes, morfologia de lesao difere muito de adulto.</li>
<li>dataset de teledermatologia real-world (foto de smartphone, nao dermoscopia profissional) com ground truth histopatologico. gap entre qualidade de imagem de pesquisa e realidade clinica.</li>
<li>dado de interacao medicamentosa em dermatologia com outcome visual. efeito colateral cutaneo de droga e documentado textualmente mas sem imaging pareado.</li>
<li>dataset de cicatrizacao/wound healing com time-series de imagem. ferida cronica (ulcera diabetica, venosa) sem tracking visual padronizado.</li>
<li>EHR psiquiatrico em lingua nao-inglesa com NLP treinado. PsyRoBERTa e dinamarques, MentalBERT e ingles, zero pra espanhol/portugues/hindi/mandarim clinico.</li>
<li>saude mental, dado de terapia longitudinal com outcome validado (nao social media). Reddit/Twitter como proxy de saude mental e cientificamente fragil.</li>
<li>genomica mais fenotipo dermatologico para populacao sub-representada (africana, indigena, mestica). GWAS dermatologico e 90%+ europeu.</li>
<li>dado de adesao a tratamento dermatologico com outcome visual. paciente comeca tratamento, abandona, piora: nenhum dataset captura esse ciclo.</li>
<li>dataset de reacao adversa cutanea a cosmetico com composicao quimica linkada. toxicovigilancia cosmetica sem dado visual em escala.</li>
<li>integracao de dado ambiental (UV, poluicao, umidade) com incidencia dermatologica geolocalizada. epidemiologia dermatologica ambiental sem dataset estruturado.</li>
</ol>
<h3 id="os-mesmos-gaps-por-dificuldade-de-execucao-mais-facil-primeiro">os mesmos gaps por dificuldade de execucao, mais facil primeiro</h3>
<ol>
<li>NLP em portugues clinico dermatologico. 6-12 meses com parceria institucional, o corpus de notas ja existe em hospital brasileiro, falta estruturar e treinar.</li>
<li>dermatologia mais saude mental (comorbidade DLQI mais PHQ-9 mais imagem). 12-24 meses, questionario validado ja existe, basta aplicar em coorte dermatologica existente.</li>
<li>dataset de teledermatologia real-world (foto smartphone mais ground truth). 12-18 meses, app de teledermatologia ja coleta essa foto, falta ground truth histopatologico.</li>
<li>dado de procedimento estetico nao-cirurgico com follow-up. 12-18 meses, clinica de estetica tem foto antes/depois em volume, falta padronizacao e consentimento.</li>
<li>dataset padronizado de estetica antes/depois com outcome measure. 12-24 meses, requer acordo com rede de clinica mas o dado bruto existe em gaveta.</li>
<li>saude mental, dado de terapia longitudinal com outcome validado. 18-24 meses, plataforma de terapia online (BetterHelp, Zenklub) tem o dado, falta acesso pra pesquisa.</li>
<li>dado de adesao a tratamento dermatologico com outcome visual. 18-24 meses, possivel em setting de clinical trial, requer protocolo prospectivo.</li>
<li>dataset de cicatrizacao/wound healing time-series. 18-24 meses, clinica de wound care ja fotografa rotineiramente, falta padronizacao e estruturacao.</li>
<li>EHR psiquiatrico em lingua nao-inglesa com NLP. 12-36 meses conforme o idioma, hospital psiquiatrico publico tem volume de notas, a barreira e regulatoria.</li>
<li>dado de reacao adversa cutanea a cosmetico com composicao quimica. 24 meses, ANVISA/FDA tem report mas sem imagem, requer coleta prospectiva.</li>
<li>integracao dado ambiental mais incidencia dermatologica geolocalizada. 24-36 meses, dado ambiental e publico (NASA, INPE), falta linkagem com registro dermatologico.</li>
<li>dermatologia pediatrica em escala. 24-36 meses, barreira etica de consentimento parental, IRB/CEP mais restritivo pra menor.</li>
<li>dado de interacao medicamentosa com outcome visual. 24-36 meses, requer pharmacovigilance prospectiva com imaging, multi-institucional.</li>
<li>dataset EHR multi-center fora do eixo EUA/Europa. 2-4 anos, requer coordenacao entre varios paises com regulacao de dado diferente.</li>
<li>diversidade Fitzpatrick IV-VI em escala (>50k imagens). 2-5 anos, requer infraestrutura de digitalizacao em regiao que nao tem, funding dedicado.</li>
<li>genomica mais fenotipo dermatologico para populacao sub-representada. 3-5 anos, requer sequenciamento mais phenotyping mais consentimento em populacao vulneravel.</li>
<li>longitudinal, lesao -> evolucao -> outcome. 3-7 anos pela natureza temporal, attrition de paciente, custo de follow-up.</li>
<li>dermatologia para tons asiaticos em escala. 3-5 anos, barreira regulatoria chinesa de exportacao de dado, infraestrutura indiana precaria.</li>
<li>WSI dermatologico aberto >10k laminas. 3-5 anos, investimento de 5-20M USD em equipamento mais anotacao, ninguem financiou em formato publico.</li>
<li>multimodal, dermoscopia mais histopatologia mais genomica linkados. 5-10 anos, requer integracao de 3+ silos hospitalares que operam de forma completamente independente, o custo de sequenciamento mais digitalizacao mais imaging por paciente deixa o custo por sample altissimo.</li>
</ol>
<h2 id="parametros-e-condicoes-de-pele-detectaveis">parametros e condicoes de pele detectaveis</h2>
<p>consolidacao dos parametros de analise de pele que apareceram nos equipamentos profissionais, organizados por categoria clinica com nota de implementacao. o que e detectavel, por qual metodo, e o que da pra fazer com hardware Apple.</p>
<h3 id="condicoes-pigmentares">condicoes pigmentares</h3>
<h4 id="manchas-spots">manchas, spots</h4>
<p>lesao pigmentar focal: sarda (efelide), lentigo solar, mancha senil (lentigo senil), nevo melanocitico. nos equipamentos: captura multi-espectral (RGB mais CPL mais UV) com algoritmo de segmentacao. implementacao: modelo de deteccao de objeto (YOLO-style ou segmentacao de instancia) treinado em dataset dermatologico (ISIC, DermNet, HAM10000). CoreML suporta as duas arquiteturas. output desejado: bounding box, mascara, classificacao por tipo, score de confianca.</p>
<h4 id="pigmentacao-geral">pigmentacao geral</h4>
<p>distribuicao total de melanina na face. diferente de mancha, que e focal, a pigmentacao geral avalia uniformidade e distribuicao. implementacao: analise no espaco LAB, mapa de intensidade do canal L* (luminancia) e b* (eixo amarelo-azul).</p>
<h4 id="hiperpigmentacao">hiperpigmentacao</h4>
<p>excesso de pigmento em area especifica. inclui melasma, PIH (hiperpigmentacao pos-inflamatoria), mascara gravidica. implementacao: classificacao binaria pixel-wise (normal vs hiperpigmentado) ou segmentacao semantica com classe hiperpigmentacao. o dado de treino tem que incluir diversidade de fototipo Fitzpatrick I a VI.</p>
<h4 id="brown-zone-melanina-profunda">brown zone, melanina profunda</h4>
<p>pigmentacao dermica profunda que nao aparece em RGB. revelada por CPL nos equipamentos. sem CPL: modelo de estimativa treinado em pares RGB/CPL, precisao inferior ao CPL real mas util como screening. com CPL (filtro clip-on): processamento de imagem direto.</p>
<h4 id="dano-solar-oculto-uv-spots">dano solar oculto, UV spots</h4>
<p>alteracao de pigmento pre-clinica de exposicao UV acumulada, visivel so sob UV. sem UV fisico, nao e detectavel direto. possivel: modelo preditivo por fator de risco (fototipo, idade, exposicao solar reportada) mais mancha visivel como proxy.</p>
<h3 id="condicoes-de-textura">condicoes de textura</h3>
<h4 id="rugas-wrinkles">rugas, wrinkles</h4>
<p>linha fina e sulco profundo. classificacao tipica: linha fina (crow’s feet, periorbital), sulco moderado (nasolabial), ruga profunda (frontal). implementacao: segmentacao semantica especializada. grading de severidade (escala de Glogau ou similar) por classificacao. o modelo tem que distinguir ruga de expressao (dinamica) de ruga estatica, idealmente com captura em face neutra.</p>
<h4 id="poros">poros</h4>
<p>abertura folicular dilatada, predominante na zona T (testa, nariz, queixo). implementacao: segmentacao em crop de alta resolucao. a resolucao minima util pra deteccao de poro fica em torno de 20 pixels por poro, o que exige crop region com magnificacao digital ou camera de alta resolucao. iPhone 48MP da conta de crop da zona T.</p>
<h4 id="textura-geral">textura geral</h4>
<p>avalia suavidade, rugosidade, uniformidade da superficie. implementacao: metrica de textura computacional (variancia local, entropia, LBP, local binary patterns) extraida de imagem em alta resolucao. pode ser output numerico (score de textura) ou mapa espacial.</p>
<h3 id="condicoes-inflamatorias-e-vasculares">condicoes inflamatorias e vasculares</h3>
<h4 id="sensibilidade-vermelhidao-difusa">sensibilidade, vermelhidao difusa</h4>
<p>pele sensivel, rosacea, irritacao. caracterizada por vermelhidao difusa nao focal. implementacao: segmentacao de area vermelha com threshold no canal a* (LAB) ou canal R (RGB). classificacao: pele normal vs sensivel vs rosacea.</p>
<h4 id="red-area-inflamacao-focal">red area, inflamacao focal</h4>
<p>eritema localizado, telangiectasia (vaso visivel), inflamacao perilesional. implementacao: deteccao de componente vermelho conectado mais classificacao de padrao (focal vs difuso vs linear/vascular).</p>
<h4 id="acne-e-inflamacao">acne e inflamacao</h4>
<p>acne comedonal (cravo), acne inflamatoria (papula, pustula), acne cistica. implementacao: deteccao de objeto multi-classe (comedao aberto, comedao fechado, papula, pustula, nodulo, cisto). dataset disponivel: ACNE04, dataset IIT Delhi.</p>
<h4 id="porfirinas-atividade-bacteriana">porfirinas, atividade bacteriana</h4>
<p>presenca de Propionibacterium acnes detectada por fluorescencia UV. requer UV fisico (365-405nm), nao simulavel computacionalmente. sem UV: acne visivel como proxy de atividade bacteriana.</p>
<h3 id="classificacoes-gerais">classificacoes gerais</h3>
<h4 id="tipo-de-pele">tipo de pele</h4>
<p>classificacao em sistema, por exemplo Fitzpatrick pra fototipo, Baumann pro tipo funcional (oleosa/seca, sensivel/resistente, pigmentada/nao-pigmentada, firme/enrugada). implementacao: questionario mais analise de imagem combinados. o questionario captura o subjetivo (sensacao de oleosidade, historico de queimadura), a imagem complementa com dado objetivo (brilho especular pra oleosidade, vermelhidao pra sensibilidade).</p>
<h4 id="grading-de-envelhecimento">grading de envelhecimento</h4>
<p>escala de severidade de envelhecimento cutaneo (Glogau, Fitzpatrick-Goldman, ou proprietaria). implementacao: regressao ou classificacao ordinal treinada em imagem facial rotulada por dermatologista. output: score numerico mais categoria (leve, moderado, severo, muito severo).</p>
<h4 id="tom-e-uniformidade">tom e uniformidade</h4>
<p>avalia uniformidade de cor (tom) da face. area de discromia, tom desigual, zona de sombra. implementacao: mapa de desvio do tom medio. calcula o tom medio da face, gera mapa de diferenca pixel-wise, destaca area com desvio acima do threshold.</p>
<h3 id="simulacoes-preditivas">simulacoes preditivas</h3>
<h4 id="simulacao-de-envelhecimento">simulacao de envelhecimento</h4>
<p>projecao visual de como a face pode envelhecer ao longo de anos. percepcao de valor alta pelo paciente, usada como motivacao pra tratamento preventivo. implementacao: modelo generativo (GAN ou diffusion) de aging facial. modelos existentes: SAM (style-based age manipulation), FADING. conversao pra CoreML via coremltools. tamanho estimado: 50 a 200MB conforme a arquitetura. rodavel na ANE (Apple Neural Engine) dos chips A15+.</p>
<h4 id="simulacao-de-evolucao-de-manchas">simulacao de evolucao de manchas</h4>
<p>projecao de como a mancha pigmentar pode progredir. implementacao: pode ser subproduto do modelo de aging, com output especifico pro canal de pigmentacao. alternativa: modelo separado mais leve focado em textura de pigmento.</p>
<h2 id="modos-de-imagem-para-analise-facial">modos de imagem para analise facial</h2>
<p>consolidacao dos modos de captura e processamento de imagem que apareceram nos equipamentos profissionais de analise facial. os equipamentos rodam de 8 a 15 modos simultaneos, com resolucoes de 20MP a 42MP full-face. dois grupos: espectro de iluminacao (depende de hardware de captura) e processamento (derivado por computacao a partir das capturas).</p>
<h3 id="espectros-de-iluminacao">espectros de iluminacao</h3>
<p>dependem de hardware de captura. e aqui que mora a maior parte da inviabilidade, porque a camera Apple so entrega RGB nativo.</p>
<h4 id="rgb-luz-visivel">RGB, luz visivel</h4>
<p>captura padrao em cor natural. base pra todo o resto. presente em 100% dos equipamentos. nativo em qualquer camera Apple, sem restricao.</p>
<h4 id="cpl-cross-polarized-light">CPL, cross-polarized light</h4>
<p>elimina a reflexao especular da superficie e revela pigmentacao subsuperficial na derme. e o modo pra ver melanina profunda, vaso sanguineo, condicao abaixo da epiderme que nao aparece no RGB. requer filtro polarizador fisico. existe clip-on de filtro pra iPhone, custo baixo. alternativa pior: modelo CoreML treinado em pares RGB/CPL pra estimar CPL a partir de RGB, com precisao reduzida.</p>
<h4 id="ppl-parallel-balanced-polarized-light">PPL, parallel/balanced polarized light</h4>
<p>enfatiza textura superficial da epiderme. complemento do CPL: onde CPL revela profundidade, PPL revela superficie. mesma situacao do CPL, requer filtro polarizador.</p>
<h4 id="uv-uva-ultravioleta">UV/UVA, ultravioleta</h4>
<p>revela dano solar invisivel a olho nu. penetra superficialmente e destaca alteracao de pigmento de exposicao solar acumulada. requer fonte UV em 365nm e sensor sensivel a UV. nao replicavel na camera Apple padrao. possivel com acessorio USB-C dedicado.</p>
<h4 id="nir-near-infrared">NIR, near-infrared</h4>
<p>penetra camadas mais profundas da derme, revela condicao vascular e estrutural profunda que luz visivel e UV nao alcancam. requer sensor IR dedicado. a TrueDepth do iPhone emite IR pro Face ID mas nao expoe o raw da imagem IR via API publica. da pra acessar por framework privado, nao recomendo pra producao.</p>
<h4 id="lampada-de-wood">lampada de Wood</h4>
<p>UV em 365nm que provoca fluorescencia em certas substancias da pele. detecta infeccao fungica, bacteriana, alteracao de pigmento. padrao em dermatologia clinica. requer hardware UV dedicado. alguns equipamentos simulam o efeito por processamento de imagem UV, mas a fluorescencia real exige excitacao fisica UV.</p>
<h3 id="modos-de-processamento">modos de processamento</h3>
<p>derivados por computacao a partir das capturas acima. todos candidatos a implementacao via CoreML, Core Image, vImage.</p>
<h4 id="sensitivity-sensitive-area">sensitivity, sensitive area</h4>
<p>mapeia area de vermelhidao difusa, sinal de pele sensivel, rosacea, irritacao. tipicamente derivado de analise do canal vermelho com segmentacao por intensidade e distribuicao espacial. implementacao: segmentacao semantica CoreML treinada em dataset de pele sensivel, ou isolamento do canal R com thresholding adaptativo via Core Image.</p>
<h4 id="red-area-red-zone">red area, red zone</h4>
<p>parecido com sensitivity mas focado em inflamacao localizada e vascularizacao visivel, telangiectasia, eritema. a diferenca tecnica: sensitivity mapeia vermelhidao difusa, red area mapeia vermelhidao focal. implementacao: isolamento do canal vermelho, deteccao de componentes conectados, classificacao de intensidade.</p>
<h4 id="heatmap">heatmap</h4>
<p>visualizacao em gradiente de cor (falso-color) mostrando distribuicao de intensidade. pode representar temperatura (se houver IR) ou intensidade cromatica (derivado de RGB). nos equipamentos analisados parece derivado de intensidade de cor, nao de temperatura real. implementacao trivial via Core Image: grayscale, aplicar color lookup table com gradiente termico.</p>
<h4 id="spots-manchas">spots, manchas</h4>
<p>detecta e segmenta mancha pigmentar: sarda, lentigo solar, melasma, mancha senil. precisa de alta resolucao pra pegar mancha pequena. implementacao: modelo de deteccao de objeto ou segmentacao de instancia (CoreML), com dataset rotulado por tipo de mancha.</p>
<h4 id="spots-simulation">spots simulation</h4>
<p>projecao computacional de como a mancha pode evoluir no tempo sem tratamento. valor percebido alto pelo paciente. implementacao: modelo generativo (GAN ou diffusion) de aging focado em pigmento, rodavel local via CoreML com modelos otimizados pra ANE.</p>
<h4 id="brown-brown-zone">brown, brown zone</h4>
<p>isola pigmentacao melanica especifica. diferente de spots, que detecta mancha individual, brown mapeia a distribuicao total de melanina. implementacao: analise de cor no espaco LAB (canal b* pra amarelo-azul, canal a* pra vermelho-verde) via Core Image. thresholding em LAB e mais robusto que RGB pra deteccao de melanina.</p>
<h4 id="hyperpigmentation">hyperpigmentation</h4>
<p>subconjunto de pigmentacao, area de pigmento excessivo. separado de pigmentacao geral, o que sugere valor clinico em distinguir pigmentacao normal de hiperpigmentacao patologica. implementacao: classificacao binaria (normal vs hiperpigmentado) por pixel ou por regiao, treinada em dado clinico rotulado.</p>
<h4 id="wrinkles-rugas">wrinkles, rugas</h4>
<p>detecta e mapeia linha fina e ruga profunda. os equipamentos usam alta resolucao mais contraste pra evidenciar textura. implementacao: deteccao de borda (Canny, Laplacian) mais segmentacao semantica especializada em linha de expressao. o Vision framework tem VNDetectFaceLandmarksRequest, retorna contorno facial mas nao e granular pra ruga individual.</p>
<h4 id="wrinkles-simulation">wrinkles simulation</h4>
<p>projecao de evolucao de ruga no tempo. complemento do spots simulation. implementacao: modelo generativo de aging facial, pode compartilhar arquitetura com spots simulation, com output separado pra pigmento e textura.</p>
<h4 id="porphyrin-porfirinas">porphyrin, porfirinas</h4>
<p>detecta atividade bacteriana (Propionibacterium acnes) por fluorescencia de porfirina sob UV. a porfirina emite fluorescencia vermelha/laranja quando excitada por UV 365-405nm. usado pra avaliar acne e atividade de glandula sebacea. requer UV real pra excitacao. nao simulavel computacionalmente a partir de RGB.</p>
<h4 id="pores-poros">pores, poros</h4>
<p>mapeia poro dilatado. requer alta resolucao e bom contraste. tipicamente analisado em crop regional (nariz, bochecha, testa), nao full-face. implementacao: segmentacao em crop de alta resolucao. a camera de 48MP do iPhone 15 Pro+ tem resolucao adequada pra deteccao de poro em crop regional.</p>
<h4 id="monochrome">monochrome</h4>
<p>conversao pra escala de cinza pra analisar contraste e textura sem influencia de cor. trivial: desaturacao via Core Image (CIColorControls).</p>
<h4 id="green-channel-canal-verde">green channel, canal verde</h4>
<p>isola o canal verde do RGB. o verde e o mais sensivel a hemoglobina e vascularizacao, da pra ver padrao vascular superficial. trivial via Core Image: extrai o canal G, renderiza em grayscale ou falso-color verde.</p>
<h4 id="uv-spots">UV spots</h4>
<p>dano solar oculto revelado por UV. mostra pigmentacao que ainda nao aparece em luz normal mas ja existe na pele (dano acumulado pre-clinico). requer UV real, nao derivavel de RGB. da pra estimar com modelo treinado em pares UV/RGB, com precisao limitada.</p>
<h3 id="calibracao-de-cor">calibracao de cor</h3>
<p>equipamento de tier superior usa calibracao com 48 cores pra garantir consistencia entre sessoes e ambientes de iluminacao. um display vertical 4K serve de referencia.</p>
<p>pro dermML: fotografar um color checker card (X-Rite ColorChecker ou similar) na mesma sessao de captura. usar Core Image pra computar a matriz de transformacao de cor que normaliza a captura pra um espaco de referencia. assim a comparacao antes/depois vale mesmo com iluminacao diferente entre sessoes.</p>
<h3 id="resolucoes-identificadas">resolucoes identificadas</h3>
<p>os equipamentos operam em 3 tiers:</p>
<table><thead><tr><th>tier</th><th>resolucao</th><th>uso</th></tr></thead><tbody>
<tr><td>1</td><td>42MP full-face</td><td>topo de linha 3D</td></tr>
<tr><td>2</td><td>20MP full-face</td><td>analisador 2D multi-espectral</td></tr>
<tr><td>3</td><td>1.3MP</td><td>microscopia capilar, magnificacao 100-200x</td></tr>
</tbody></table>
<p>camera iPhone 15 Pro+: 48MP no sensor main, 12MP no ultra-wide/macro. a resolucao nativa e competitiva ou superior ao tier 1 e 2, mas sem controle de iluminacao multi-espectral integrado. essa e a fronteira: o sensor da conta, o que falta e o espectro.</p>
<h2 id="modelagem-3d-facial">modelagem 3D facial</h2>
<p>capacidade de captura e reconstrucao 3D facial dos equipamentos profissionais, e o que da pra alcancar com hardware Apple. resumo da fronteira: a precisao submilimetrica dos equipamentos dedicados e inatingivel via Apple, mas pro uso estetico o ARKit/LiDAR resolve.</p>
<h3 id="capacidades-dos-equipamentos">capacidades dos equipamentos</h3>
<p>reconstrucao 3D de alta precisao: resolucao 42MP full-face, precisao geometrica 0.2mm (submilimetrica). aplicacao em avaliacao estetica tridimensional, planejamento cirurgico, tracking de resultado de procedimento.</p>
<p>visualizacao 3D de textura: renderiza a superficie da pele com textura de alta resolucao mapeada. rotacao e zoom interativo pra examinar varios angulos. presente no tier intermediario (20MP) com reconstrucao 3D parcial.</p>
<p>comparacao 3D: comparacao volumetrica de captura em momentos diferentes. quantifica alteracao de volume (preenchimento, perda de gordura facial, edema).</p>
<h3 id="implementacao-via-apple">implementacao via Apple</h3>
<h4 id="arkit-face-tracking-truedepth">ARKit face tracking, TrueDepth</h4>
<p>disponivel em iPhone com Face ID (iPhone X+). retorna face mesh com 1220 vertices e blend shapes em tempo real. precisao geometrica inferior a 0.2mm, estimada em 1 a 3mm pro contorno facial. limitacao: so funciona na camera frontal (TrueDepth), nao na traseira, exceto iPhone com LiDAR.</p>
<h4 id="lidar-scanner-ipad-pro-iphone-pro">LiDAR scanner, iPad Pro, iPhone Pro</h4>
<p>disponivel em iPad Pro (2020+) e iPhone 12 Pro+. retorna depth map e mesh 3D via ARKit. precisao 1 a 5mm conforme a distancia. vantagem: funciona na camera traseira com controle de iluminacao. uso: captura 3D facial com iluminacao controlada mais textura de alta resolucao pela camera 48MP em simultaneo.</p>
<h4 id="scenekit-realitykit-pra-renderizacao">SceneKit / RealityKit pra renderizacao</h4>
<p>renderizacao interativa do mesh 3D capturado com textura mapeada. rotacao, zoom, medicao interativa. export em USDZ (formato nativo Apple) pra compartilhamento e visualizacao em AR.</p>
<h4 id="photogrammetry-api-object-capture">Photogrammetry API, Object Capture</h4>
<p>disponivel desde macOS Monterey / iOS 17. reconstroi objeto 3D a partir de varias fotos 2D. precisao depende do numero de foto e da angulacao. uso potencial: captura 3D facial a partir de sequencia de foto (video de rotacao da face). nao ideal pro uso clinico pela variabilidade, mas util pra tracking longitudinal.</p>
<h3 id="consideracoes-para-dermml">consideracoes para dermML</h3>
<p>a precisao de 0.2mm dos equipamentos dedicados e inatingivel via hardware Apple atual. pro uso clinico (planejamento cirurgico), precisa de hardware dedicado. pro uso estetico (tracking de envelhecimento, avaliacao de simetria, visualizacao pro paciente), a precisao do ARKit/LiDAR e suficiente.</p>
<p>o diferencial do dermML seria juntar a captura 3D (mesmo com precisao inferior) com analise de textura via CoreML. nenhum equipamento analisado faz analise de condicao de pele direto no modelo 3D, eles capturam 3D e analisam textura em 2D separado. o dermML podia mapear o resultado da analise 2D (mancha, ruga, poro) direto sobre o mesh 3D, numa visualizacao integrada que nenhum deles tem.</p>
<h2 id="otimizacao-de-compressao-av1-por-ordenacao-visual-de-frames">otimizacao de compressao AV1 por ordenacao visual de frames</h2>
<p>proposta de otimizacao pro sistema AV1-backed dataset do dermML: ordenar a imagem por similaridade visual antes do encoding pra maximizar a compressao inter-frame. autor Brenner Cruvinel, Hoff Research LTDA. fevereiro 2026, proposta, nao implementado. esse e o sistema base que virou o spin-off [[truw]], onde o truque do AV1 foi generalizado e tambem onde foi falsificado como index exato.</p>
<h3 id="1-problema-atual">1. problema atual</h3>
<p>no encoding atual a imagem e empacotada como frame AV1 na ordem do filesystem (ordem alfabetica do nome de arquivo). isso quer dizer que frame adjacente no video pode ser visualmente muito diferente:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="plain"><span class="giallo-l"><span>Frame 0: ISIC_0024306.jpg (melanoma escuro, pele clara)</span></span>
<span class="giallo-l"><span>Frame 1: ISIC_0024307.jpg (nevo benigno, pele escura)</span></span>
<span class="giallo-l"><span>Frame 2: ISIC_0024308.jpg (dermatofibroma, pele media)</span></span></code></pre>
<p>o codec AV1 usa predicao inter-frame: pra codificar o frame N ele calcula o delta em relacao ao frame N-1 (ou outro frame de referencia). se o frame adjacente e visualmente distinto, o delta e grande e a compressao inter-frame se perde.</p>
<p>o resultado atual (~30x de compressao) vem quase inteiro da eficiencia intra-frame do AV1 (compressao dentro de cada imagem). a compressao inter-frame esta subutilizada.</p>
<h3 id="2-proposta-ordenacao-por-similaridade-visual">2. proposta, ordenacao por similaridade visual</h3>
<p>antes de encodar a imagem como video, ordenar de forma que frame adjacente seja maximamente similar. assim o delta inter-frame fica minimo e o AV1 comprime muito mais agressivo.</p>
<h4 id="2-1-metodo-tsp-nos-embeddings-dermlip">2.1 metodo, TSP nos embeddings DermLIP</h4>
<p>os embeddings DermLIP (512 dimensoes, L2-normalizados) ja existem no pipeline de indexacao. a ordenacao otima e equivalente ao travelling salesman problem (TSP) no espaco de embedding: achar a sequencia que minimiza a distancia total entre frame consecutivo. pra 58k pontos o TSP exato e intratavel. heuristica viavel:</p>
<p>nearest neighbor greedy (mais simples):</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="python"><span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> numpy</span><span style="color: light-dark(#859900, #859900);"> as</span><span> np</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">from</span><span> sklearn</span><span>.</span><span>metrics</span><span>.</span><span>pairwise</span><span style="color: light-dark(#859900, #859900);"> import</span><span> cosine_distances</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">def</span><span style="color: light-dark(#268BD2, #268BD2);"> sort_by_visual_similarity</span><span>(</span><span>embeddings</span><span>,</span><span> image_paths</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> Ordena imagens por similaridade visual usando nearest-neighbor greedy.</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> embeddings: np.array shape (N, 512), L2-normalized</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> image_paths: list of str, paths das imagens</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> Retorna: indices ordenados</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span></span>
<span class="giallo-l"><span> N</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#268BD2, #268BD2);"> len</span><span>(</span><span>embeddings</span><span>)</span></span>
<span class="giallo-l"><span> visited</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#859900, #859900);"> set</span><span>(</span><span>)</span></span>
<span class="giallo-l"><span> order</span><span style="color: light-dark(#859900, #859900);"> =</span><span> [</span><span>]</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span> current</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#D33682, #D33682);"> 0</span></span>
<span class="giallo-l"><span> visited</span><span>.</span><span>add</span><span>(</span><span>current</span><span>)</span></span>
<span class="giallo-l"><span> order</span><span>.</span><span>append</span><span>(</span><span>current</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> _</span><span style="color: light-dark(#859900, #859900);"> in</span><span style="color: light-dark(#268BD2, #268BD2);"> range</span><span>(</span><span>N</span><span style="color: light-dark(#859900, #859900);"> -</span><span style="color: light-dark(#D33682, #D33682);"> 1</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span> dists</span><span style="color: light-dark(#859900, #859900);"> =</span><span> cosine_distances</span><span>(</span><span>embeddings</span><span>[</span><span>current</span><span>:</span><span>current</span><span style="color: light-dark(#859900, #859900);">+</span><span style="color: light-dark(#D33682, #D33682);">1</span><span>]</span><span>,</span><span> embeddings</span><span>)</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">0</span><span>]</span></span>
<span class="giallo-l"><span> dists</span><span>[</span><span style="color: light-dark(#859900, #859900);">list</span><span>(</span><span>visited</span><span>)</span><span>]</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#859900, #859900);"> float</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#2AA198, #2AA198);">inf</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>)</span></span>
<span class="giallo-l"><span> nearest</span><span style="color: light-dark(#859900, #859900);"> =</span><span> np</span><span>.</span><span>argmin</span><span>(</span><span>dists</span><span>)</span></span>
<span class="giallo-l"><span> visited</span><span>.</span><span>add</span><span>(</span><span>nearest</span><span>)</span></span>
<span class="giallo-l"><span> order</span><span>.</span><span>append</span><span>(</span><span>nearest</span><span>)</span></span>
<span class="giallo-l"><span> current</span><span style="color: light-dark(#859900, #859900);"> =</span><span> nearest</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> return</span><span> order</span></span></code></pre>
<p>complexidade O(N^2) em distancia. pra 58k imagens com embedding 512-dim isso leva ~10-30 minutos em CPU. aceitavel como processo offline.</p>
<p>FAISS accelerated (recomendado pra dataset grande):</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="python"><span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> faiss</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> numpy</span><span style="color: light-dark(#859900, #859900);"> as</span><span> np</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">def</span><span style="color: light-dark(#268BD2, #268BD2);"> sort_by_similarity_faiss</span><span>(</span><span>embeddings</span><span>,</span><span> image_paths</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> Versao acelerada usando FAISS para nearest-neighbor.</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> Usa busca aproximada para datasets > 50k.</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span></span>
<span class="giallo-l"><span> N</span><span>,</span><span> dim</span><span style="color: light-dark(#859900, #859900);"> =</span><span> embeddings</span><span>.</span><span>shape</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span> index</span><span style="color: light-dark(#859900, #859900);"> =</span><span> faiss</span><span>.</span><span>IndexFlatIP</span><span>(</span><span>dim</span><span>)</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> Inner product = cosine em vetores normalizados</span></span>
<span class="giallo-l"><span> index</span><span>.</span><span>add</span><span>(</span><span>embeddings</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span> visited</span><span style="color: light-dark(#859900, #859900);"> =</span><span> np</span><span>.</span><span>zeros</span><span>(</span><span>N</span><span>,</span><span> dtype</span><span style="color: light-dark(#859900, #859900);">=</span><span style="color: light-dark(#859900, #859900);">bool</span><span>)</span></span>
<span class="giallo-l"><span> order</span><span style="color: light-dark(#859900, #859900);"> =</span><span> np</span><span>.</span><span>zeros</span><span>(</span><span>N</span><span>,</span><span> dtype</span><span style="color: light-dark(#859900, #859900);">=</span><span>np</span><span>.</span><span>int64</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span> current</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#D33682, #D33682);"> 0</span></span>
<span class="giallo-l"><span> visited</span><span>[</span><span>current</span><span>]</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#B58900, #B58900);"> True</span></span>
<span class="giallo-l"><span> order</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">0</span><span>]</span><span style="color: light-dark(#859900, #859900);"> =</span><span> current</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> step</span><span style="color: light-dark(#859900, #859900);"> in</span><span style="color: light-dark(#268BD2, #268BD2);"> range</span><span>(</span><span style="color: light-dark(#D33682, #D33682);">1</span><span>,</span><span> N</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span> k</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#268BD2, #268BD2);"> min</span><span>(</span><span>step</span><span style="color: light-dark(#859900, #859900);"> +</span><span style="color: light-dark(#D33682, #D33682);"> 100</span><span>,</span><span> N</span><span>)</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> Buscar mais que o necessario para ter margem</span></span>
<span class="giallo-l"><span> scores</span><span>,</span><span> indices</span><span style="color: light-dark(#859900, #859900);"> =</span><span> index</span><span>.</span><span>search</span><span>(</span><span>embeddings</span><span>[</span><span>current</span><span>:</span><span>current</span><span style="color: light-dark(#859900, #859900);">+</span><span style="color: light-dark(#D33682, #D33682);">1</span><span>]</span><span>,</span><span> k</span><span>)</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> idx</span><span style="color: light-dark(#859900, #859900);"> in</span><span> indices</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">0</span><span>]</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> if</span><span style="color: light-dark(#859900, #859900);"> not</span><span> visited</span><span>[</span><span>idx</span><span>]</span><span>:</span></span>
<span class="giallo-l"><span> current</span><span style="color: light-dark(#859900, #859900);"> =</span><span> idx</span></span>
<span class="giallo-l"><span> visited</span><span>[</span><span>current</span><span>]</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#B58900, #B58900);"> True</span></span>
<span class="giallo-l"><span> order</span><span>[</span><span>step</span><span>]</span><span style="color: light-dark(#859900, #859900);"> =</span><span> current</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> break</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> return</span><span> order</span><span>.</span><span>tolist</span><span>(</span><span>)</span></span></code></pre>
<p>clustering mais sort (alternativa hierarquica):</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="python"><span class="giallo-l"><span style="color: light-dark(#859900, #859900);">from</span><span> sklearn</span><span>.</span><span>cluster</span><span style="color: light-dark(#859900, #859900);"> import</span><span> KMeans</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> numpy</span><span style="color: light-dark(#859900, #859900);"> as</span><span> np</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">def</span><span style="color: light-dark(#268BD2, #268BD2);"> sort_by_clustering</span><span>(</span><span>embeddings</span><span>,</span><span> n_clusters</span><span style="color: light-dark(#859900, #859900);">=</span><span style="color: light-dark(#D33682, #D33682);">100</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> Agrupa imagens em clusters, ordena clusters por centroide,</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> ordena imagens dentro de cada cluster por distancia ao centroide.</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> Resultado: transicoes suaves entre grupos visuais.</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span></span>
<span class="giallo-l"><span> kmeans</span><span style="color: light-dark(#859900, #859900);"> =</span><span> KMeans</span><span>(</span><span>n_clusters</span><span style="color: light-dark(#859900, #859900);">=</span><span>n_clusters</span><span>,</span><span> random_state</span><span style="color: light-dark(#859900, #859900);">=</span><span style="color: light-dark(#D33682, #D33682);">42</span><span>)</span></span>
<span class="giallo-l"><span> labels</span><span style="color: light-dark(#859900, #859900);"> =</span><span> kmeans</span><span>.</span><span>fit_predict</span><span>(</span><span>embeddings</span><span>)</span></span>
<span class="giallo-l"><span> centroids</span><span style="color: light-dark(#859900, #859900);"> =</span><span> kmeans</span><span>.</span><span>cluster_centers_</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span> cluster_order</span><span style="color: light-dark(#859900, #859900);"> =</span><span> sort_by_visual_similarity</span><span>(</span><span>centroids</span><span>,</span><span style="color: light-dark(#859900, #859900);"> list</span><span>(</span><span style="color: light-dark(#268BD2, #268BD2);">range</span><span>(</span><span>n_clusters</span><span>)</span><span>)</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span> final_order</span><span style="color: light-dark(#859900, #859900);"> =</span><span> [</span><span>]</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> cluster_idx</span><span style="color: light-dark(#859900, #859900);"> in</span><span> cluster_order</span><span>:</span></span>
<span class="giallo-l"><span> mask</span><span style="color: light-dark(#859900, #859900);"> =</span><span> labels</span><span style="color: light-dark(#859900, #859900);"> ==</span><span> cluster_idx</span></span>
<span class="giallo-l"><span> indices</span><span style="color: light-dark(#859900, #859900);"> =</span><span> np</span><span>.</span><span>where</span><span>(</span><span>mask</span><span>)</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">0</span><span>]</span></span>
<span class="giallo-l"><span> dists</span><span style="color: light-dark(#859900, #859900);"> =</span><span> np</span><span>.</span><span>linalg</span><span>.</span><span>norm</span><span>(</span><span>embeddings</span><span>[</span><span>indices</span><span>]</span><span style="color: light-dark(#859900, #859900);"> -</span><span> centroids</span><span>[</span><span>cluster_idx</span><span>]</span><span>,</span><span> axis</span><span style="color: light-dark(#859900, #859900);">=</span><span style="color: light-dark(#D33682, #D33682);">1</span><span>)</span></span>
<span class="giallo-l"><span> sorted_indices</span><span style="color: light-dark(#859900, #859900);"> =</span><span> indices</span><span>[</span><span>np</span><span>.</span><span>argsort</span><span>(</span><span>dists</span><span>)</span><span>]</span></span>
<span class="giallo-l"><span> final_order</span><span>.</span><span>extend</span><span>(</span><span>sorted_indices</span><span>.</span><span>tolist</span><span>(</span><span>)</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> return</span><span> final_order</span></span></code></pre><h4 id="2-2-pipeline-completo-proposto">2.2 pipeline completo proposto</h4>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="plain"><span class="giallo-l"><span>1. carregar todas as imagens do dataset</span></span>
<span class="giallo-l"><span>2. gerar embeddings DermLIP por imagem (ja existe no pipeline atual)</span></span>
<span class="giallo-l"><span>3. ordenar indices por similaridade visual (nearest-neighbor greedy ou clustering)</span></span>
<span class="giallo-l"><span>4. gerar filelist.txt com a imagem na ordem otimizada</span></span>
<span class="giallo-l"><span>5. ffmpeg encode AV1 com a lista ordenada</span></span>
<span class="giallo-l"><span>6. indexar frames com mapeamento: frame_idx -> (imagem original, posicao no dataset)</span></span></code></pre>
<p>script proposto, <code>sort_and_encode.py</code>:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="python"><span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">!/usr/bin/env python3</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);">"""</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);">Ordena imagens por similaridade visual e encoda como AV1.</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);">Maximiza compressao inter-frame.</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);">"""</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> numpy</span><span style="color: light-dark(#859900, #859900);"> as</span><span> np</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> faiss</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">from</span><span> pathlib</span><span style="color: light-dark(#859900, #859900);"> import</span><span> Path</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> subprocess</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">import</span><span> tempfile</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">def</span><span style="color: light-dark(#268BD2, #268BD2);"> load_embeddings</span><span>(</span><span>index_path</span><span>,</span><span> db_path</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span><span style="color: light-dark(#2AA198, #2AA198);">Carrega embeddings e metadata existentes.</span><span style="color: light-dark(#2AA198, #2AA198);">"""</span></span>
<span class="giallo-l"><span> index</span><span style="color: light-dark(#859900, #859900);"> =</span><span> faiss</span><span>.</span><span>read_index</span><span>(</span><span style="color: light-dark(#859900, #859900);">str</span><span>(</span><span>index_path</span><span>)</span><span>)</span></span>
<span class="giallo-l"><span> embeddings</span><span style="color: light-dark(#859900, #859900);"> =</span><span> faiss</span><span>.</span><span>rev_swig_ptr</span><span>(</span></span>
<span class="giallo-l"><span> index</span><span>.</span><span>get_xb</span><span>(</span><span>)</span><span>,</span><span> index</span><span>.</span><span>ntotal</span><span style="color: light-dark(#859900, #859900);"> *</span><span> index</span><span>.</span><span>d</span></span>
<span class="giallo-l"><span> )</span><span>.</span><span>reshape</span><span>(</span><span>index</span><span>.</span><span>ntotal</span><span>,</span><span> index</span><span>.</span><span>d</span><span>)</span><span>.</span><span>copy</span><span>(</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> import</span><span> sqlite3</span></span>
<span class="giallo-l"><span> conn</span><span style="color: light-dark(#859900, #859900);"> =</span><span> sqlite3</span><span>.</span><span>connect</span><span>(</span><span style="color: light-dark(#859900, #859900);">str</span><span>(</span><span>db_path</span><span>)</span><span>)</span></span>
<span class="giallo-l"><span> conn</span><span>.</span><span>row_factory</span><span style="color: light-dark(#859900, #859900);"> =</span><span> sqlite3</span><span>.</span><span>Row</span></span>
<span class="giallo-l"><span> rows</span><span style="color: light-dark(#859900, #859900);"> =</span><span> conn</span><span>.</span><span>execute</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">SELECT * FROM frames ORDER BY idx</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span><span>.</span><span>fetchall</span><span>(</span><span>)</span></span>
<span class="giallo-l"><span> metadata</span><span style="color: light-dark(#859900, #859900);"> =</span><span> [</span><span style="color: light-dark(#859900, #859900);">dict</span><span>(</span><span>r</span><span>)</span><span style="color: light-dark(#859900, #859900);"> for</span><span> r</span><span style="color: light-dark(#859900, #859900);"> in</span><span> rows</span><span>]</span></span>
<span class="giallo-l"><span> conn</span><span>.</span><span>close</span><span>(</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> return</span><span> embeddings</span><span>,</span><span> metadata</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">def</span><span style="color: light-dark(#268BD2, #268BD2);"> greedy_nearest_neighbor</span><span>(</span><span>embeddings</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span><span style="color: light-dark(#2AA198, #2AA198);">Ordena por nearest-neighbor greedy no espaco de embeddings.</span><span style="color: light-dark(#2AA198, #2AA198);">"""</span></span>
<span class="giallo-l"><span> N</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#268BD2, #268BD2);"> len</span><span>(</span><span>embeddings</span><span>)</span></span>
<span class="giallo-l"><span> index</span><span style="color: light-dark(#859900, #859900);"> =</span><span> faiss</span><span>.</span><span>IndexFlatIP</span><span>(</span><span>embeddings</span><span>.</span><span>shape</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">1</span><span>]</span><span>)</span></span>
<span class="giallo-l"><span> index</span><span>.</span><span>add</span><span>(</span><span>embeddings</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span> visited</span><span style="color: light-dark(#859900, #859900);"> =</span><span> np</span><span>.</span><span>zeros</span><span>(</span><span>N</span><span>,</span><span> dtype</span><span style="color: light-dark(#859900, #859900);">=</span><span style="color: light-dark(#859900, #859900);">bool</span><span>)</span></span>
<span class="giallo-l"><span> order</span><span style="color: light-dark(#859900, #859900);"> =</span><span> [</span><span>]</span></span>
<span class="giallo-l"><span> current</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#D33682, #D33682);"> 0</span></span>
<span class="giallo-l"><span> visited</span><span>[</span><span>current</span><span>]</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#B58900, #B58900);"> True</span></span>
<span class="giallo-l"><span> order</span><span>.</span><span>append</span><span>(</span><span>current</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> step</span><span style="color: light-dark(#859900, #859900);"> in</span><span style="color: light-dark(#268BD2, #268BD2);"> range</span><span>(</span><span style="color: light-dark(#D33682, #D33682);">1</span><span>,</span><span> N</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span> scores</span><span>,</span><span> indices</span><span style="color: light-dark(#859900, #859900);"> =</span><span> index</span><span>.</span><span>search</span><span>(</span><span>embeddings</span><span>[</span><span>current</span><span>:</span><span>current</span><span style="color: light-dark(#859900, #859900);">+</span><span style="color: light-dark(#D33682, #D33682);">1</span><span>]</span><span>,</span><span style="color: light-dark(#268BD2, #268BD2);"> min</span><span>(</span><span>N</span><span>,</span><span style="color: light-dark(#D33682, #D33682);"> 200</span><span>)</span><span>)</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> idx</span><span style="color: light-dark(#859900, #859900);"> in</span><span> indices</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">0</span><span>]</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> if</span><span style="color: light-dark(#859900, #859900);"> not</span><span> visited</span><span>[</span><span>idx</span><span>]</span><span>:</span></span>
<span class="giallo-l"><span> current</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#859900, #859900);"> int</span><span>(</span><span>idx</span><span>)</span></span>
<span class="giallo-l"><span> visited</span><span>[</span><span>current</span><span>]</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#B58900, #B58900);"> True</span></span>
<span class="giallo-l"><span> order</span><span>.</span><span>append</span><span>(</span><span>current</span><span>)</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> break</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> return</span><span> order</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">def</span><span style="color: light-dark(#268BD2, #268BD2);"> generate_sorted_filelist</span><span>(</span><span>metadata</span><span>,</span><span> order</span><span>,</span><span> output_path</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span><span style="color: light-dark(#2AA198, #2AA198);">Gera filelist.txt na ordem otimizada para ffmpeg concat.</span><span style="color: light-dark(#2AA198, #2AA198);">"""</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> with</span><span style="color: light-dark(#268BD2, #268BD2);"> open</span><span>(</span><span>output_path</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">w</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>)</span><span style="color: light-dark(#859900, #859900);"> as</span><span> f</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> idx</span><span style="color: light-dark(#859900, #859900);"> in</span><span> order</span><span>:</span></span>
<span class="giallo-l"><span> meta</span><span style="color: light-dark(#859900, #859900);"> =</span><span> metadata</span><span>[</span><span>idx</span><span>]</span></span>
<span class="giallo-l"><span> image_path</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;"> f</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">datasets/</span><span style="color: light-dark(#CB4B16, #CB4B16);">{</span><span>meta</span><span>[</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#2AA198, #2AA198);">dataset</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>]</span><span style="color: light-dark(#CB4B16, #CB4B16);">}</span><span style="color: light-dark(#2AA198, #2AA198);">/</span><span style="color: light-dark(#CB4B16, #CB4B16);">{</span><span>meta</span><span>[</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#2AA198, #2AA198);">original_image_id</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>]</span><span style="color: light-dark(#CB4B16, #CB4B16);">}</span><span style="color: light-dark(#2AA198, #2AA198);">"</span></span>
<span class="giallo-l"><span> f</span><span>.</span><span>write</span><span>(</span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">f</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">file </span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#CB4B16, #CB4B16);">{</span><span>image_path</span><span style="color: light-dark(#CB4B16, #CB4B16);">}</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#CB4B16, #CB4B16);">\n</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> return</span><span> output_path</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">def</span><span style="color: light-dark(#268BD2, #268BD2);"> encode_av1</span><span>(</span><span>filelist_path</span><span>,</span><span> output_path</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> """</span><span style="color: light-dark(#2AA198, #2AA198);">Encoda video AV1 a partir da lista ordenada.</span><span style="color: light-dark(#2AA198, #2AA198);">"""</span></span>
<span class="giallo-l"><span> cmd</span><span style="color: light-dark(#859900, #859900);"> =</span><span> [</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">ffmpeg</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-y</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-f</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">concat</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-safe</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">0</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-r</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">1</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-i</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#859900, #859900);"> str</span><span>(</span><span>filelist_path</span><span>)</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-c:v</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">libsvtav1</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-crf</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">30</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-pix_fmt</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">yuv420p</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">-r</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> '</span><span style="color: light-dark(#2AA198, #2AA198);">1</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> str</span><span>(</span><span>output_path</span><span>)</span></span>
<span class="giallo-l"><span> ]</span></span>
<span class="giallo-l"><span> subprocess</span><span>.</span><span>run</span><span>(</span><span>cmd</span><span>,</span><span> check</span><span style="color: light-dark(#859900, #859900);">=</span><span style="color: light-dark(#B58900, #B58900);">True</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">if</span><span> __name__</span><span style="color: light-dark(#859900, #859900);"> ==</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">__main__</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> INDEX_PATH</span><span style="color: light-dark(#859900, #859900);"> =</span><span> Path</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">outputteste/compressed/video_embeddings.index</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> DB_PATH</span><span style="color: light-dark(#859900, #859900);"> =</span><span> Path</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">outputteste/compressed/video_metadata.db</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> print</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">[1/4] Carregando embeddings...</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"><span> embeddings</span><span>,</span><span> metadata</span><span style="color: light-dark(#859900, #859900);"> =</span><span> load_embeddings</span><span>(</span><span style="color: light-dark(#CB4B16, #CB4B16);">INDEX_PATH</span><span>,</span><span style="color: light-dark(#CB4B16, #CB4B16);"> DB_PATH</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> print</span><span>(</span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">f</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">[2/4] Ordenando </span><span style="color: light-dark(#CB4B16, #CB4B16);">{</span><span style="color: light-dark(#268BD2, #268BD2);">len</span><span>(</span><span>embeddings</span><span>)</span><span style="color: light-dark(#CB4B16, #CB4B16);">}</span><span style="color: light-dark(#2AA198, #2AA198);"> imagens por similaridade visual...</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"><span> order</span><span style="color: light-dark(#859900, #859900);"> =</span><span> greedy_nearest_neighbor</span><span>(</span><span>embeddings</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> print</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">[3/4] Gerando filelist ordenada...</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"><span> filelist</span><span style="color: light-dark(#859900, #859900);"> =</span><span> generate_sorted_filelist</span><span>(</span><span>metadata</span><span>,</span><span> order</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">outputteste/sorted_filelist.txt</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> print</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">[4/4] Encoding AV1 com ordem otimizada...</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"><span> encode_av1</span><span>(</span><span>filelist</span><span>,</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">outputteste/compressed/dataset_sorted_av1.mp4</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);"> print</span><span>(</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">[OK] Encoding completo.</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>)</span></span></code></pre><h3 id="3-estimativa-de-ganho-de-compressao">3. estimativa de ganho de compressao</h3>
<h4 id="3-1-raciocinio-teorico">3.1 raciocinio teorico</h4>
<p>no encoding atual (ordem de filesystem) a maior parte da compressao vem do modo intra-frame do AV1. a inter-frame contribui pouco porque frame adjacente e imagem de paciente/lesao completamente diferente.</p>
<p>com ordenacao por similaridade: frame adjacente tem background de pele similar (mesmo tom, iluminacao), lesao adjacente tem formato/cor/textura similar, o delta inter-frame fica dominado por pequena variacao local. em codec de video a eficiencia inter-frame tipicamente adiciona 2-5x de compressao sobre o modo intra-only pra conteudo com alta redundancia temporal.</p>
<h4 id="3-2-estimativa-conservadora">3.2 estimativa conservadora</h4>
<table><thead><tr><th>cenario</th><th>compressao estimada</th></tr></thead><tbody>
<tr><td>atual (ordem filesystem)</td><td>~30x</td></tr>
<tr><td>com ordenacao por similaridade</td><td>~50-80x</td></tr>
<tr><td>com ordenacao mais tuning de CRF/GOP</td><td>~60-100x</td></tr>
</tbody></table>
<p>pro dataset completo: atual 52 GB -> ~1.7 GB, estimado com ordenacao 52 GB -> ~650 MB a 1.0 GB.</p>
<h4 id="3-3-como-validar">3.3 como validar</h4>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 1. encodar o mesmo dataset com e sem ordenacao</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">ffmpeg</span><span style="color: light-dark(#2AA198, #2AA198);"> ...</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">i</span><span style="color: light-dark(#2AA198, #2AA198);"> filelist_original.txt</span><span style="color: light-dark(#2AA198, #2AA198);"> ...</span><span style="color: light-dark(#2AA198, #2AA198);"> dataset_unsorted.mp4</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">ffmpeg</span><span style="color: light-dark(#2AA198, #2AA198);"> ...</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">i</span><span style="color: light-dark(#2AA198, #2AA198);"> filelist_sorted.txt</span><span style="color: light-dark(#2AA198, #2AA198);"> ...</span><span style="color: light-dark(#2AA198, #2AA198);"> dataset_sorted.mp4</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 2. comparar tamanhos</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">ls</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">lh</span><span style="color: light-dark(#2AA198, #2AA198);"> dataset_unsorted.mp4</span><span style="color: light-dark(#2AA198, #2AA198);"> dataset_sorted.mp4</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 3. comparar qualidade (VMAF, PSNR, SSIM)</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">ffmpeg</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">i</span><span style="color: light-dark(#2AA198, #2AA198);"> dataset_unsorted.mp4</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">i</span><span style="color: light-dark(#2AA198, #2AA198);"> dataset_sorted.mp4</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">lavfi</span><span style="color: light-dark(#2AA198, #2AA198);"> libvmaf</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">f</span><span style="color: light-dark(#2AA198, #2AA198);"> null</span><span style="color: light-dark(#2AA198, #2AA198);"> -</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 4. verificar que a busca por similaridade funciona igual</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> (re-indexar o video sorted e comparar resultados de query)</span></span></code></pre><h3 id="4-consideracoes-de-implementacao">4. consideracoes de implementacao</h3>
<h4 id="4-1-mapeamento-frame-imagem-original">4.1 mapeamento frame -> imagem original</h4>
<p>com a ordenacao, o frame_idx no video sorted nao corresponde mais ao indice original. o SQLite metadata precisa de uma coluna extra:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="sql"><span class="giallo-l"><span style="color: light-dark(#859900, #859900);">CREATE</span><span style="color: light-dark(#859900, #859900);"> TABLE</span><span style="color: light-dark(#268BD2, #268BD2);"> frames_sorted</span><span> (</span></span>
<span class="giallo-l"><span> idx </span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">INTEGER</span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;"> PRIMARY KEY</span><span>, </span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">--</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> indice no FAISS</span></span>
<span class="giallo-l"><span> sorted_frame_idx </span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">INTEGER</span><span>, </span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">--</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> posicao no video sorted</span></span>
<span class="giallo-l"><span> video_file </span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">TEXT</span><span>,</span></span>
<span class="giallo-l"><span> original_frame_idx </span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">INTEGER</span><span>, </span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">--</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> posicao no video original</span></span>
<span class="giallo-l"><span> dataset </span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">TEXT</span><span>,</span></span>
<span class="giallo-l"><span> original_image_id </span><span style="color: light-dark(#586E75, #93A1A1);font-weight: bold;">TEXT</span></span>
<span class="giallo-l"><span>);</span></span></code></pre><h4 id="4-2-gop-structure">4.2 GOP structure</h4>
<p>pra maximizar compressao inter-frame, configurar o GOP (group of pictures) do SVT-AV1:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">ffmpeg</span><span style="color: light-dark(#2AA198, #2AA198);"> ...</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">c:v</span><span style="color: light-dark(#2AA198, #2AA198);"> libsvtav1</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">svtav1-params</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">keyint=250:scd=0</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> ...</span></span></code></pre>
<p><code>keyint=250</code> poe keyframe a cada 250 frames (permite inter-prediction por trecho longo). <code>scd=0</code> desabilita scene change detection (nao quero keyframe extra quando a lesao muda). com GOP longo, o seek aleatorio pra um frame especifico exige decodar desde o keyframe anterior. pra thumbnail sob demanda o custo e ~10-100ms extra por seek. aceitavel.</p>
<h4 id="4-3-estrategias-de-ordenacao-alternativas">4.3 estrategias de ordenacao alternativas</h4>
<p>Hilbert curve no espaco de embedding: projeta o embedding 512-dim numa curva de Hilbert 1-dim, preserva localidade espacial melhor que nearest-neighbor greedy, mais uniforme, evita cluster isolado.</p>
<p>spectral ordering: usa o segundo autovetor do Laplaciano do grafo de similaridade (Fiedler vector) pra achar a ordenacao que minimiza a soma total de diferenca entre vizinho. matematicamente otimo, mas O(N^2) em memoria pra matriz de afinidade.</p>
<p>hierarchical clustering (dendrogram traverse): constroi arvore hierarquica de cluster e percorre a folha em ordem, garante transicao suave em multiplas escalas.</p>
<p>pra uma primeira implementacao, nearest-neighbor greedy ou clustering mais sort dao conta e sao simples.</p>
<h3 id="5-potencial-como-publicacao-ou-ferramenta-open-source">5. potencial como publicacao ou ferramenta open-source</h3>
<h4 id="5-1-contribuicao-original">5.1 contribuicao original</h4>
<p>ate onde foi verificado (fevereiro 2026), nao existe publicacao ou ferramenta que combine: 1. codec de video (AV1) como formato de armazenamento de dataset de imagem, 2. ordenacao por similaridade visual pra maximizar compressao inter-frame, 3. indexacao de embedding por frame pra busca integrada, 4. benchmark comparativo sistematico de codec de video como formato de dataset.</p>
<p>cada elemento existe isolado: AV1 como compressor de imagem (AVIF), ordenacao por similaridade (TSP em embedding), FAISS pra busca, dataset empacotado (TFRecord, WebDataset). a combinacao desses elementos num pipeline integrado pra dataset medico e, ate onde verificado, original.</p>
<h4 id="5-2-formato-proposto-videodataset">5.2 formato proposto, VideoDataset</h4>
<p>especificacao pra um formato de dataset baseado em video:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="plain"><span class="giallo-l"><span>dataset.vds/</span></span>
<span class="giallo-l"><span> data.mp4 # imagens como frames AV1 (ordenadas por similaridade)</span></span>
<span class="giallo-l"><span> embeddings.index # FAISS index (opcional, para busca)</span></span>
<span class="giallo-l"><span> metadata.db # SQLite: frame_idx -> metadados originais</span></span>
<span class="giallo-l"><span> manifest.json # metadados do dataset (schema, contagem, versao, codec params)</span></span></code></pre>
<p>manifest.json:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="json"><span class="giallo-l"><span>{</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">format</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">videodataset</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">version</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">1.0</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">codec</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">av1</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">encoder</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">libsvtav1</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">total_frames</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 58000</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">resolution</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">512x512</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">fps</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 1</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">crf</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 30</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">sorting</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">dermlip-nearest-neighbor</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">embedding_model</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">redlessone/DermLIP_ViT-B-16</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">embedding_dim</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 512</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">original_size_gb</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 52.0</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">compressed_size_gb</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 0.65</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">compression_ratio</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 80</span><span>,</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">datasets</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span> [</span></span>
<span class="giallo-l"><span> {</span><span style="color: light-dark(#859900, #859900);">"</span><span style="color: light-dark(#859900, #859900);">name</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">HAM10000</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">frames</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 10015</span><span>,</span><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">start_idx</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 0</span><span>}</span><span>,</span></span>
<span class="giallo-l"><span> {</span><span style="color: light-dark(#859900, #859900);">"</span><span style="color: light-dark(#859900, #859900);">name</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">DermNet</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">frames</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 19446</span><span>,</span><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">start_idx</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 10015</span><span>}</span><span>,</span></span>
<span class="giallo-l"><span> {</span><span style="color: light-dark(#859900, #859900);">"</span><span style="color: light-dark(#859900, #859900);">name</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">ISIC2019</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>,</span><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">frames</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 25331</span><span>,</span><span style="color: light-dark(#859900, #859900);"> "</span><span style="color: light-dark(#859900, #859900);">start_idx</span><span style="color: light-dark(#859900, #859900);">"</span><span>:</span><span style="color: light-dark(#D33682, #D33682);"> 29461</span><span>}</span></span>
<span class="giallo-l"><span> ]</span></span>
<span class="giallo-l"><span>}</span></span></code></pre><h4 id="5-3-possivel-titulo-de-paper">5.3 possivel titulo de paper</h4>
<p>“VideoDataset: AV1 Video-Backed Compression for Medical Image Datasets with Integrated Similarity Search”. venue relevante: MICCAI (Medical Image Computing and Computer Assisted Intervention), ML4H (Machine Learning for Health), NeurIPS Datasets and Benchmarks Track, Nature Scientific Data, arXiv cs.CV / cs.MM.</p>
<h4 id="5-4-pontos-fortes-para-review">5.4 pontos fortes para review</h4>
<p>pratico: resolve um problema real de engenharia (distribuicao de dataset de 50+ GB). reprodutivel: usa ferramenta existente (ffmpeg, FAISS, DermLIP). benchmark rigoroso: 7 formatos comparados empiricamente. generalizavel: funciona pra qualquer dataset de imagem, nao so dermatologia. royalty-free: AV1 e codec aberto, sem licenciamento.</p>
<h2 id="lerobotdataset-v3-0-large-scale-datasets-in-lerobot">LeRobotDataset v3.0, large-scale datasets in lerobot</h2>
<p>clipping de referencia. Hugging Face, 16 de setembro de 2025, autores Francesco Capuano, Michel Aractingi, Quentin Lhoest, Caroline Pascal, Pepijn Kooijmans, Jade Choghari, Remi Cadene, Simon Alibert, Adil Zouitine, Martino Russi, Steven Palma. fonte https://huggingface.co/blog/lerobot-datasets-v3 . relevante pro dermML como prior art de formato: o v3 empacota varios episodios num arquivo so com metadata relacional, e suporta streaming. o benchmark de codec esta em [[scaling-robotics-datasets-video-encoding]].</p>
<h3 id="resumo">resumo</h3>
<p>o v2 guardava um episodio por arquivo, batendo no limite de file-system ao escalar pra milhoes de episodio. o v3 empacota varios episodios num arquivo so, usando metadata relacional pra recuperar a informacao no nivel do episodio individual. o formato novo suporta nativo o acesso em modo streaming, processando dataset grande on the fly. tem util de uma linha pra converter todo dataset do formato LeRobotDataset pro novo.</p>
<p>o LeRobotDataset e formato padronizado pras necessidades de robot learning, com acesso unificado a dado de robotica em varias modalidades (leitura sensorimotor, varios feed de camera, status de teleoperacao). tambem guarda informacao geral de como o dado foi coletado (metadata), incluindo descricao textual da tarefa, tipo de robo e detalhe de medicao como o frame rate em que a imagem e o estado do robo sao amostrados. a metadata serve pra indexar e buscar dataset de robotica no Hub.</p>
<h3 id="instalar-lerobot-e-gravar-dataset">instalar lerobot e gravar dataset</h3>
<p>o v3 vai ser parte do lerobot a partir do <code>lerobot-v0.4.0</code>. da pra instalar a ultima <code>lerobot-v0.3.x</code> que suporta o formato direto do PyPI:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">pip</span><span style="color: light-dark(#2AA198, #2AA198);"> install</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">https://github.com/huggingface/lerobot/archive/33cad37054c2b594ceba57463e8f11ee374fa93c.zip</span><span style="color: light-dark(#2AA198, #2AA198);">"</span></span></code></pre>
<p>gravar dataset com o braco SO-101 por teleoperacao:</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">lerobot-record</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-robot.type=so101_follower</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-robot.port=/dev/tty.usbmodem585A0076841</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-robot.id=my_awesome_follower_arm</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-robot.cameras=</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">{ front: {type: opencv, index_or_path: 0, width: 1920, height: 1080, fps: 30}}</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-teleop.type=so101_leader</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-teleop.port=/dev/tty.usbmodem58760431551</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-teleop.id=my_awesome_leader_arm</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-display_data=true</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-dataset.repo_id=</span><span style="color: light-dark(#859900, #859900);">$</span><span style="color: light-dark(#859900, #859900);">{</span><span style="color: light-dark(#268BD2, #268BD2);">HF_USER</span><span style="color: light-dark(#859900, #859900);">}</span><span style="color: light-dark(#CB4B16, #CB4B16);">/record-test</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-dataset.num_episodes=5</span><span style="color: light-dark(#CB4B16, #CB4B16);"> \</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-dataset.single_task=</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#2AA198, #2AA198);">Grab the black cube</span><span style="color: light-dark(#2AA198, #2AA198);">"</span></span></code></pre><h3 id="design-do-formato">design do formato</h3>
<p>a escolha de design central e separar o storage subjacente da API de usuario. o dataset organiza em tres componentes:</p>
<ol>
<li>tabular data: dado de baixa dimensao e alta frequencia (estado de junta, acao) em arquivo Apache Parquet, tipicamente offloaded pra biblioteca <code>datasets</code>, com acesso memory-mapped ou por streaming.</li>
<li>visual data: pra lidar com volume grande de dado de camera, o frame e concatenado e encodado em arquivo MP4. frame do mesmo episodio sempre agrupado no mesmo video, varios video agrupados por camera. pra reduzir o stress no file system, grupo de video da mesma camera tambem e quebrado em subdiretorio.</li>
<li>metadata: colecao de arquivo JSON que descreve a estrutura do dataset, contraparte relacional do tabular e do visual. inclui schema de feature, frame rate, estatistica de normalizacao, fronteira de episodio.</li>
</ol>
<p>pra suportar dataset com milhoes de episodio (centenas de milhoes ou bilhoes de frame), o dado de episodio diferente e merged na mesma estrutura. qualquer colecao tabular e qualquer video nao tem so um episodio, e a concatenacao de varios. a metadata recupera a info especifica do episodio (timestamp de inicio e fim de um episodio num video).</p>
<p>estrutura do repositorio:</p>
<ul>
<li><code>meta/info.json</code>: arquivo central de metadata, schema completo do dataset, todas as features (<code>observation.state</code>, <code>action</code>), shapes, tipos. tambem guarda fps, versao da codebase, e os path templates pra localizar arquivo de dado e de video.</li>
<li><code>meta/stats.json</code>: estatistica agregada (mean, std, min, max) por feature em todo o dataset, usada pra normalizacao, acessivel via <code>dataset.meta.stats</code>.</li>
<li><code>meta/tasks.jsonl</code>: mapeia descricao de tarefa em linguagem natural pra indice inteiro de tarefa, usado em policy training condicionado por tarefa.</li>
<li><code>meta/episodes/</code>: metadata de cada episodio individual (length, tarefa correspondente, ponteiro de onde o dado fica). pra escalar, guardado em Parquet chunked, nao num JSON grande.</li>
<li><code>data/</code>: dado tabular frame a frame em Parquet, dado de varios episodios concatenado em arquivo maior, organizado em subdiretorio chunked.</li>
<li><code>videos/</code>: arquivo MP4 de todo stream de observacao visual, video de varios episodios concatenado em MP4 unico, reduzindo o numero de arquivo. a estrutura de path (<code>/videos/<camera_key>/<chunk>/file_...mp4</code>) deixa o data loader localizar o arquivo certo e fazer seek pro timestamp do frame.</li>
</ul>
<h3 id="migrar-do-v2-1-pro-v3-0">migrar do v2.1 pro v3.0</h3>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="shellscript"><span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">python</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">m</span><span style="color: light-dark(#2AA198, #2AA198);"> lerobot.datasets.v30.convert_dataset_v21_to_v30</span><span style="color: light-dark(#CB4B16, #CB4B16);"> -</span><span style="color: light-dark(#CB4B16, #CB4B16);">-repo-id=</span><span style="color: light-dark(#859900, #859900);"><</span><span style="color: light-dark(#CB4B16, #CB4B16);">HFUSER/DATASET_ID</span><span style="color: light-dark(#859900, #859900);">></span></span></code></pre>
<p>o script <code>convert_dataset_v21_to_v30.py</code> agrega os varios episodios <code>episode-0000.mp4, episode-0001.mp4, ...</code> / <code>episode-0000.parquet, ...</code> em arquivo unico <code>file-0000.mp4</code> / <code>file-0000.parquet</code>, e atualiza a metadata pra recuperar a info especifica do episodio dos arquivos de nivel mais alto.</p>
<h3 id="exemplo-com-torch-dataloader">exemplo com torch DataLoader</h3>
<p>todo dataset no Hub com os tres pilares (tabular, visual, metadata) acessa com uma linha. a maioria dos algoritmo de robot learning (RL ou behavioral cloning) opera num stack de observacao e acao. RL tipicamente usa historico de observacao previa <code>o_{t-H_o:t}</code>, BC tipicamente regressa chunk de varias acao. o LeRobotDataset tem operacao nativa de windowing pelo argumento <code>delta_timestamps</code>.</p>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="python"><span class="giallo-l"><span style="color: light-dark(#859900, #859900);">from</span><span> lerobot</span><span>.</span><span>datasets</span><span>.</span><span>lerobot_dataset</span><span style="color: light-dark(#859900, #859900);"> import</span><span> LeRobotDataset</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>repo_id</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">yaak-ai/L2D-v3</span><span style="color: light-dark(#2AA198, #2AA198);">"</span></span>
<span class="giallo-l"><span>dataset</span><span style="color: light-dark(#859900, #859900);"> =</span><span> LeRobotDataset</span><span>(</span><span>repo_id</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>sample</span><span style="color: light-dark(#859900, #859900);"> =</span><span> dataset</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">100</span><span>]</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> {</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 'observation.state': tensor([...]),</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 'action': tensor([...]),</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 'observation.images.front_left': tensor([C, H, W]),</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 'timestamp': tensor(1.234),</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> }</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>delta_timestamps</span><span style="color: light-dark(#859900, #859900);"> =</span><span> {</span></span>
<span class="giallo-l"><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">observation.images.front_left</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span>:</span><span> [</span><span style="color: light-dark(#859900, #859900);">-</span><span style="color: light-dark(#D33682, #D33682);">0.2</span><span>,</span><span style="color: light-dark(#859900, #859900);"> -</span><span style="color: light-dark(#D33682, #D33682);">0.1</span><span>,</span><span style="color: light-dark(#D33682, #D33682);"> 0.0</span><span>]</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> #</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 0.2 e 0.1 seg antes da observacao</span></span>
<span class="giallo-l"><span>}</span></span>
<span class="giallo-l"><span>dataset</span><span style="color: light-dark(#859900, #859900);"> =</span><span> LeRobotDataset</span><span>(</span><span>repo_id</span><span>,</span><span> delta_timestamps</span><span style="color: light-dark(#859900, #859900);">=</span><span>delta_timestamps</span><span>)</span></span>
<span class="giallo-l"><span>sample</span><span style="color: light-dark(#859900, #859900);"> =</span><span> dataset</span><span>[</span><span style="color: light-dark(#D33682, #D33682);">100</span><span>]</span></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> 'observation.images.front_left' agora tem shape [T, C, H, W], T=3</span></span>
<span class="giallo-l"><span style="color: light-dark(#268BD2, #268BD2);">print</span><span>(</span><span>sample</span><span>[</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#2AA198, #2AA198);">observation.images.front_left</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>]</span><span>.</span><span>shape</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>batch_size</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#D33682, #D33682);"> 16</span></span>
<span class="giallo-l"><span>data_loader</span><span style="color: light-dark(#859900, #859900);"> =</span><span> torch</span><span>.</span><span>utils</span><span>.</span><span>data</span><span>.</span><span>DataLoader</span><span>(</span><span>dataset</span><span>,</span><span> batch_size</span><span style="color: light-dark(#859900, #859900);">=</span><span>batch_size</span><span>)</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>num_epochs</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#D33682, #D33682);"> 1</span></span>
<span class="giallo-l"><span>device</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">cuda</span><span style="color: light-dark(#2AA198, #2AA198);">"</span><span style="color: light-dark(#859900, #859900);"> if</span><span> torch</span><span>.</span><span>cuda</span><span>.</span><span>is_available</span><span>(</span><span>)</span><span style="color: light-dark(#859900, #859900);"> else</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">cpu</span><span style="color: light-dark(#2AA198, #2AA198);">"</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);">for</span><span> epoch</span><span style="color: light-dark(#859900, #859900);"> in</span><span style="color: light-dark(#268BD2, #268BD2);"> range</span><span>(</span><span>num_epochs</span><span>)</span><span>:</span></span>
<span class="giallo-l"><span style="color: light-dark(#859900, #859900);"> for</span><span> batch</span><span style="color: light-dark(#859900, #859900);"> in</span><span> data_loader</span><span>:</span></span>
<span class="giallo-l"><span> observations</span><span style="color: light-dark(#859900, #859900);"> =</span><span> batch</span><span>[</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#2AA198, #2AA198);">observation.state.vehicle</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>]</span><span>.</span><span>to</span><span>(</span><span>device</span><span>)</span></span>
<span class="giallo-l"><span> actions</span><span style="color: light-dark(#859900, #859900);"> =</span><span> batch</span><span>[</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#2AA198, #2AA198);">action.continuous</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>]</span><span>.</span><span>to</span><span>(</span><span>device</span><span>)</span></span>
<span class="giallo-l"><span> images</span><span style="color: light-dark(#859900, #859900);"> =</span><span> batch</span><span>[</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span style="color: light-dark(#2AA198, #2AA198);">observation.images.front_left</span><span style="color: light-dark(#2AA198, #2AA198);">'</span><span>]</span><span>.</span><span>to</span><span>(</span><span>device</span><span>)</span></span>
<span class="giallo-l"><span style="color: light-dark(#CB4B16, #CB4B16);"> ...</span></span></code></pre><h3 id="streaming">streaming</h3>
<pre class="giallo" style="color-scheme: light dark; color: light-dark(#657B83, #839496); background-color: light-dark(#FDF6E3, #002B36);"><code data-lang="python"><span class="giallo-l"><span style="color: light-dark(#859900, #859900);">from</span><span> lerobot</span><span>.</span><span>datasets</span><span>.</span><span>streaming_dataset</span><span style="color: light-dark(#859900, #859900);"> import</span><span> StreamingLeRobotDataset</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;">#</span><span style="color: light-dark(#93A1A1, #586E75);font-style: italic;"> Streams direto do Hub, sem baixar nem carregar na memoria</span></span>
<span class="giallo-l"><span>repo_id</span><span style="color: light-dark(#859900, #859900);"> =</span><span style="color: light-dark(#2AA198, #2AA198);"> "</span><span style="color: light-dark(#2AA198, #2AA198);">yaak-ai/L2D-v3</span><span style="color: light-dark(#2AA198, #2AA198);">"</span></span>
<span class="giallo-l"><span>dataset</span><span style="color: light-dark(#859900, #859900);"> =</span><span> StreamingLeRobotDataset</span><span>(</span><span>repo_id</span><span>)</span></span></code></pre><h3 id="conclusao-1">conclusao</h3>
<p>o v3.0 e um passo pra escalar dataset de robotica no LeRobot. fornecendo um formato pra guardar e acessar colecao grande de dado de robo, permite a comunidade treinar em potencialmente milhoes de episodio sem nem baixar o dado. o time agradece o time da yaak.ai pelo suporte no desenvolvimento.</p>
<h3 id="contribuicao-do-lerobotdataset">contribuicao do LeRobotDataset</h3>
<p>reducao media de tamanho: 14% do tamanho original do dataset (ate 0.2% no melhor caso). melhora de loading: frame unico comparavel ao PNG, frame multiplo sucessivo 25% a 50% do tempo de PNG. preservacao de qualidade: capacidade de treino mantida. ferramenta de visualizacao: browsing facil do dataset.</p>
<h3 id="fundamentos-de-video-encoding">fundamentos de video encoding</h3>
<p>duas tecnicas principais: compressao espacial (mesmo principio do JPEG/PNG, explora auto-similaridade dentro do frame) e compressao temporal (guarda a diferenca entre frame em vez do frame inteiro, exige keyframe I-frame em intervalo regular como ponto de referencia).</p>
<p>processo de encoding: 1. determina keyframe pela especificacao do usuario e pela mudanca de cena, 2. comprime keyframe espacialmente, 3. comprime a diferenca inter-frame (P-frame, B-frame) temporalmente, 4. aplica compressao espacial nessa diferenca, 5. encoda o dado comprimido em bitstream, 6. empacota o bitstream em container (MP4, MKV, AVI), 7. aplica processamento extra pra reduzir distorcao visual.</p>
<h3 id="criterios-de-avaliacao">criterios de avaliacao</h3>
<p>quatro criterios: tamanho (impacta storage e download), tempo de decodificacao (impacta tempo de treino), qualidade (impacta acuracia de treino), compatibilidade (capacidade de decodar e visualizar em varios device/plataforma).</p>
<p>metrica de tamanho: razao de compressao = tamanho do video encodado dividido pelo tamanho dos frames originais nao comprimidos. tempo de loading: tempo de decodar frame dividido pelo tempo de carregar de imagem individual. metrica de qualidade: MSE (mean square error, menor melhor), PSNR (peak signal-to-noise ratio, maior melhor), SSIM (structural similarity index measure, varia de -1 a 1, maior melhor).</p>
<h3 id="variaveis-testadas">variaveis testadas</h3>
<p>datasets (4 exemplos representativos): <code>lerobot/pusht_image</code> 96x96 px simulacao formas simples, <code>aliberts/aloha_mobile_shrimp_image</code> 480x640 px real indoor camera movel, <code>aliberts/paris_street</code> 720x1280 px real outdoor camera movel, <code>aliberts/kitchen</code> 1080x1920 px real indoor camera fixa.</p>
<p>parametros de encoding:</p>
<table><thead><tr><th>parametro</th><th>valores</th></tr></thead><tbody>
<tr><td>vcodec</td><td><code>libx264</code>, <code>libx265</code>, <code>libsvtav1</code></td></tr>
<tr><td>pix_fmt</td><td><code>yuv444p</code>, <code>yuv420p</code></td></tr>
<tr><td>g (GOP size)</td><td>1, 2, 3, 4, 5, 6, 10, 15, 20, 40, None</td></tr>
<tr><td>crf</td><td>0, 5, 10, 15, 20, 25, 30, 40, 50, None</td></tr>
</tbody></table>
<p>vcodec e o motor algoritmico do encoding. pix_fmt especifica o color space (YUV, RGB, grayscale); pra YUV, o chroma subsampling (<code>yuv420p</code> = 4:2:0, mais compativel com web; <code>yuv444p</code> info de cor completa mas menos compativel com browser). g determina a frequencia de keyframe (valor menor = mais keyframe = acesso aleatorio a frame mais rapido; valor maior = acesso mais lento mas arquivo menor; pra treino de ML precisa de valor menor; midia tradicional aceita 2-4 segundos entre keyframe). crf controla a quantidade de compressao com perda (0 = sem perda, 50-60 = muito lossy; preferivel a targeting de bitrate porque mantem qualidade visual constante com bitrate variavel).</p>
<p>decoder testado: <code>pyav</code> (default), <code>video_reader</code>. cenario de timestamp: <code>1_frame</code>, <code>2_frames</code>, <code>6_frames</code>, <code>2_frames_4_space</code>.</p>
<h3 id="resultados">resultados</h3>
<h4 id="mudanca-de-versao">mudanca de versao</h4>
<table><thead><tr><th>metrica</th><th>v1.5</th><th>v1.6</th></tr></thead><tbody>
<tr><td>vcodec</td><td><code>libx264</code></td><td><code>libsvtav1</code></td></tr>
<tr><td>pix-fmt</td><td><code>yuv444p</code></td><td><code>yuv420p</code></td></tr>
<tr><td>g</td><td>2</td><td>2</td></tr>
<tr><td>crf</td><td>None (=23)</td><td>30</td></tr>
</tbody></table>
<p>melhora: melhor qualidade com encoding AV1 e melhor compatibilidade com <code>yuv420p</code>.</p>
<h4 id="tamanho">tamanho</h4>
<p>razao de compressao media ~14% em todos os datasets. maioria <40% do tamanho original. melhor caso <1% do tamanho original. variacao por formato original (imagem nao comprimida comprime melhor), imagem ja comprimida (JPEG/PNG comprime menos), resolucao, complexidade de cena. exemplo de reducao: <code>lerobot/nyu_rot_dataset</code> 5.3MB -> 318.2KB (5.8%), <code>lerobot/aloha_sim_transfer_cube_human</code> 17.9GB -> 66.7MB (0.4%), <code>lerobot/berkeley_gnm_recon</code> 18.7GB -> 29.3MB (0.2%).</p>
<h4 id="tempo-de-loading">tempo de loading</h4>
<p>video loading escala melhor com resolucao, principalmente pra multiplos frame. pra small (96x96) e large (1080x1920): frame unico comparavel ao PNG, 2 e 6 consecutivos 25-50% do tempo de PNG.</p>
<h4 id="qualidade-g-2-crf-30">qualidade (g=2, crf=30)</h4>
<p>pra <code>lerobot/kitchen</code> (1080x1920, 2.07 megapixels):</p>
<table><thead><tr><th>codec</th><th>format</th><th>MSE</th><th>PSNR</th><th>SSIM</th></tr></thead><tbody>
<tr><td>libx264</td><td>yuv420p</td><td>2.32E-04</td><td>36.77</td><td>95.47%</td></tr>
<tr><td>libx264</td><td>yuv444p</td><td>2.06E-04</td><td>37.38</td><td>95.58%</td></tr>
<tr><td>libx265</td><td>yuv420p</td><td>6.87E-04</td><td>35.27</td><td>95.11%</td></tr>
<tr><td>libx265</td><td>yuv444p</td><td>6.75E-04</td><td>35.50</td><td>95.13%</td></tr>
<tr><td>libsvtav1</td><td>yuv420p</td><td>1.32E-04</td><td>39.20</td><td>96.84%</td></tr>
</tbody></table>
<p>libsvtav1 com yuv420p entrega a melhor metrica de qualidade.</p>
<h4 id="validacao-de-treino-de-policy">validacao de treino de policy</h4>
<p>policy treinada em dataset encodado teve performance equivalente a treinada na versao de imagem. diffusion policy no PushT: curva de treino identica entre formato, sem degradacao. ACT policy no ALOHA: curva identica, sem degradacao. AV1 vs H264: diffusion no pusht AV1 aprox H264, ACT no aloha_sim_transfer_cube_human AV1 aprox H264, ACT no aloha_sim_insertion_scripted AV1 aprox H264.</p>
<h3 id="future-work">future work</h3>
<p>parametro de encoding a explorar: <code>-preset</code> (trade-off velocidade/compressao), <code>-tune</code> (otimizar pra aspecto especifico, film quality, live, fast decode), two-pass encoding (aumenta qualidade ao custo de tempo de encoding). decoder alternativo: <code>torchcodec</code>, <code>torchaudio</code>, <code>ffmpegio</code>, <code>decord</code>, <code>nvc</code>. outras consideracoes: video encoding com depth map, otimizacao de parametro de codec, exploracao de outro container.</p>
<h3 id="conclusao-2">conclusao</h3>
<p>o benchmark demonstra que o codec AV1 com pixel format yuv420p e CRF conservador da o melhor balanco de tamanho, qualidade, velocidade e compatibilidade pra dataset de robotica. o video encoding resolve a escalabilidade de dataset de robotica reduzindo storage pra ~14% do original em media, mantendo ou melhorando a velocidade de loading pra multiplos frame, preservando a qualidade de treino e a performance da policy.</p>
<p>future</p>