I asked the Pixel 9 Pro’s AI to write a review about itself. Here’s what it said

Google’s new Pixel 9 Pro and Pro XL have a lot to offer, from the revamped design to the fancy new AI capabilities. And while many of those capabilities rely on images—including the wild generative AI Pixel Studio uses—Google’s Gemini Advanced can provide in-depth (if sometimes inaccurate) answers to almost all of your questions.

So I thought I’d give it the ultimate challenge: testing itself. I’ve already written my own review of the Pixel 9 Pro XL, highlighting good things like the updated design, long software support, and good daylight photography skills, but also less positive aspects like the monthly subscription to Gemini Advanced and disappointing low-light image quality.

My review is based on my 13 years as a technology journalist specializing in mobile devices, as well as my experience testing the phone in a variety of real-world scenarios. The Pixel’s review is presumably based on information it can gather from the internet. I asked it to “write a 200-word review of the Pixel 9 Pro XL.” Here’s the result:

“The Pixel 9 Pro XL is Google’s biggest and boldest smartphone yet. With a sleek new design, beautiful 6.8-inch display, and powerful Tensor G4 chip, it’s a true flagship. The camera system is exceptional, capturing stunning photos and videos in any lighting. Battery life is impressive too, easily lasting a full day on a single charge.

“However, it’s not perfect. The phone is quite expensive and the lack of expandable storage could be a deal-breaker for some. Also, while the Tensor G4 is fast, it doesn’t quite match the raw performance of some rivals.

“Overall, the Pixel 9 Pro XL is a fantastic phone with a lot to offer. If you’re looking for a top-of-the-line Android device with an incredible camera and long battery life, it’s definitely worth considering.”

Honestly? It’s not a bad attempt. Like me, it points out the “sleek new design” and even admits that the phone is expensive and the Tensor G4 processor isn’t up to scratch. However, it pushes the camera’s capabilities way beyond what I consider to be appropriate, making it read more like marketing material than an honest, experience-based review.

While the example above is a verbatim quote from one of the responses I received, I asked it to test the Pixel 9 Pro XL many times, varying my wording each time. Interestingly, the results changed quite drastically.

Some trials pointed to poor macro photography, while others bemoaned inconsistent battery life. One review concluded, “If you’re looking for the absolute best in performance or design, other options may be a better fit.” Another noted, “The design feels stale and the price is high for what you get,” and called the AI ​​features “gimmicky.”

The results were inconsistent, and every time I ran the query, a different opinion was generated. This is simply the nature of AI search; factors such as where the information comes from or how the algorithm puts it together are not consistent. That’s why if you asked it 100 times to write a poem about a horse in the circus, you would get 100 different poems.

But beyond simple inconsistencies, I also noticed factual errors. Some reviews incorrectly mentioned the Tensor G3 processor (instead of the Tensor G4 actually used), and there were other errors in key specifications. Inaccurate information is something I found more generally in my testing of Gemini Advanced. For example, the service suggested I listen to songs that simply don’t exist and directed me to a bar that’s long since closed to get a beer.

The lesson? Be careful how much trust you put in AI-generated answers. Of course, I’ll tell you to trust CNET’s experts more than an AI when you’re looking to buy a product, but this applies more generally. Trust your doctor more than AI-generated medical advice; trust a real electrician when it comes to redoing your home’s wiring; and definitely try to use your own common sense when an AI is trying to tell you to eat glue and rocks.