Why Does the Quality Persist in Such a Poor State?
Evaluation of Omega Testing
Despite being in open beta for nearly a year, Google’s Seek AI continues to produce confusing and often inappropriate comments.
The AI-driven enhancement to the software, known as the Search Generative Experience (SGE), still presents inaccurate or false information, as highlighted by the Washington Post’s assessment of Google’s SGE introduced last May.
Moreover, SGE employs a feature known as imagination, a technical term for generating content, even though it no longer promotes absurd statements like melting eggs or glorifying slavery. In a test search for a fictional Chinese restaurant named “Danny’s Dan Dan Noodles” in San Francisco, the results included references to long queues and excessive wait times, along with misleading information about 4,000-person lines and a two-year waiting list.
However, the issue with SGE goes beyond fabrications. Even when sourcing information from legitimate websites, the reliability of these sources is often questionable. As per an evaluation by the marketing system SE Ranking, Quora, the popular question-and-answer platform responsible for the infamous egg-melting response, emerged as the most frequently referenced website by the AI-informed search beta.
Inaccuracies in Financial Data
SGE also struggles with accurately presenting financial figures.
For instance, while a standard Google search correctly listed Meta CEO Mark Zuckerberg’s net worth as \(169.8 billion USD, SGE provided a perplexing breakdown when posed with the same query: "\)46.24 per hour, or $96,169 per year”.
The response continued with inconsistent calculations, stating figures like “\(8,014 per month, \)1,849 per week, and $230.6 million per day”, which do not align logically when combined.
Some of the sources utilized by the system appear to be outdated or have been removed entirely. In a recent search for Zuckerberg’s net worth, the result displayed $170.3 billion USD without the option to delve deeper using generative AI.
While Google asserts that SGE is an opt-in feature, reports from Search Engine Land suggest that the generative AI function is being defaulted for users, especially for complex queries or those requiring information from multiple sources.
As the one-year mark approaches, it is evident that SGE still grapples with numerous issues, raising concerns about Google’s gradual integration of this technology into broader search outcomes.
