🤖DataAgent: DVA
Last updated
Last updated
Gata is building the DataAgent Platform, enabling individuals to run various DataAgents that automatically generate cutting-edgee data and earn passive rewards.
DVA (Data Validation Agent) is Gata's first DataAgent. It runs locally to assess the quality of image-caption pairs from the whole internet, assigning a score between -1 and 1. DVA scores help select the highest-quality image-caption data, which are then used to train various vision-language AIs, such as Stable Diffusion, DALL-E, and GPT-4o.
Internet-scale image-caption datasets, such as LAION and DataComp, underpin the pre-training of all vision-language AIs. These datasets, comprising tens of billions of image-caption pairs scraped from the internet, power text-to-image generative AI models like Stable Diffusion and DALL-E, as well as vision-language models like GPT-4V and GPT-4o.However, the AI industry faces two major challenges with internet-scale image-caption datasets:
Exhaustion of Publicly Accessible Data: Most publicly available image-caption data has already been utilized.
Noisy Captions: Captions derived from web scraping often fail to accurately represent the corresponding images.
Both academia and industry have explored two key approaches to address these challenges:
Filtering: By evaluating the quality of image-caption pairs and assigning scores, high-quality subsets can be filtered for better training, as demonstrated in Apple's recent work.
Synthetic Caption Generation: Vision-language models can be used to generate more accurate captions, significantly enhancing dataset quality, another approach explored by Apple.
DVA evaluates the quality of image-caption pairs from across the internet and assigns scores ranging from -1 to 1. It is based on the Data Filtering Networks. These scores enable the selection of the highest-quality data, exemplifying the filtering approach.
Q: Why can I only process one job every 45 seconds? A: Although the jobs are computationally light, we enforce a 45-second limit to prevent any one user from monopolizing DataAgent jobs and to ensure everyone can participate.
Q: I've run jobs for a while and completed some, but I haven't earned any Intelligence points. Why? A: Intelligence points are earned only when the majority of peers running the same job agree on the result through our consensus mechanism. This process can take days, and even correct executions might not earn points if the peers disagree.
Q: I've noticed that only 10% of my completed jobs have earned intelligence points. Why? A: This is expected during our beta phase. We have set a high consensus threshold to ensure quality, which means fewer jobs earn points at this time. Several factors may contribute to this, including malicious actors exploiting the DataAgent by not completing tasks authentically and occasional machine-related execution errors. We appreciate your understanding and are actively working to increase the points awarded once beta testing is complete.
