You could always argue that people authorized to work in the U.S. won't take the jobs because they are paid at very low wages, working conditions are poor, etc.
If you want to pay 6 dollars for an avocado, though, you can keep up with your arguments. I know it's not an ideal situation, far from it. But the farming industry is already quite subsidized by the government and it needs to be competitive enough to flood Mexico's market with corn and beans, so we can't really raise wages back home.
Plus, what would happen to the restaurant industry (remember, our economy as a nation and in California is largely service-based)?. If the cost of food goes up for businesses, they will serve their meals at higher prices and pay their workers even less than they already do. Basically, low-wage labor protects what we have left of certain job markets.
I'm no economist, I could be wrong. Please point stuff out to me. I'm just writing about what I've read and talked to people about.
And even if it was well-paid, let's say, $15-$20 an hour (remember, this is "unskilled" labor), do you think authorized workers would flock to California's hot Central Valley and pick strawberries for 13 hours a day? Not so sure. Maybe. Who knows.