You have brought up a good point, actually.
If you are talking about photovoltaic solar power, yes the many large batteries that are usually used to store the electricity to feed to the inverters represent a lot of pollution, especially in their manufacture, as does the manufacture of the panels themselves. Both components have a limited lifespan, and unless totally recycled will cause yet more pollution.
However, having said that, if the solar power is put into use for a long period, then its benefit would outweigh the degree of pollution in comparison to conventional fossil fuel power conversion. This especially true if the solar power is directly converted to AC and fed directly to the grid during daylight hours.
This equally applies to wind power (another kind of pollution - visual) unless the turbines are synched and can feed direct to the grid, without the use of storage batteries and inverters..
In the case of mirrors feeding heat to boilers and steam turbines, the pollution of course is the steam, the water vapor byproduct which, as we all know by now, constitutes 95% of the gases which cause the greenhouse effect (love it or hate it). This is a serious downside to this technology.
Until the technology is perfected to convert heat directly to electricity efficiently, the problem continues. Just like coal/oil powered and nuclear powered stations we seem to be stuck with steam pollution. The only way of mitigating it might be to feed it out to sea and expel it deep underwater, not really possible unless coastally located..