The intent of affirmative action is to correct for past prejudice. The intent of equal opportunity is to correct current prejudice. You often hear about employers advertising themselves as "Equal Opportunity / Affirmative Action" employers. The trouble with these two goals is that they are in conflict with each other. The goal of equal opportunity is to have a society with no prejudice: where all individuals are evaluated on their own merits. Affirmative action (AA, henceforth), on the other hand, requires employers and educators to treat the harmed individuals specially.
Logically, they are incompatible. There is only one way this situation can be saved: if AA is strictly limited in time. AA is a law that must eventually go away once the harm has been substantially addressed. Or the addressed groups must be limited as one group's harm is compensated but another group's remains (e.g. blacks have been under AA since the beginning, but the disabled were added later.)
It's really important that AA have a goal in sight. Prejudice is generally regarded as counter-factual. Let's say that you are prejudiced against blacks; you think that blacks make worse accountants. You would prefer to hire a white accountant. Prior to AA, it's likely that a black accountant would have had to work harder in school, in order to overcome the racism of those who think blacks would make bad accountants. So the racist's prejudice would be exactly backwards.
If AA is goes on longer than it should, then you end up with the opposite situation. Rather than blacks being given a hand up to the level of whites, blacks are effectively told "Our expectations of you are lower," "You can't do as well as whites, so we have AA for you," and "You don't have to work for success." Since a black can get into a degree-granting program with lower credentials, graduate with lower grades, and be hired by an accounting firm under AA, the racist has a concrete reason for preferring white accountants to blacks.
As reparation, AA is perfectly fine. "We harmed you in the past; this makes up for it." But reparation beyond the extent of the damage becomes a crutch. The question at hand is not "should we have AA?" but instead "has AA done its job; if so we must abolish it to avoid creating harm."