Department under fire
James Pavitt has worked for the agency for 31 years, five as the deputy director of operations.
His identity had been unknown until last April when, in an unprecedented move, he appeared publicly before the 11 September commission.
At the time he said the failures that occurred before the attacks were due to woefully inadequate resources, not a lack of caring.
The BBC's Ian Pannell, in Washington, says it is his department's record in gathering intelligence in Iraq that has come in for the strongest criticism.
In particular they are criticised for not having enough good human intelligence on the ground, that they placed too much credence on badly sourced material.
A spokesman for the CIA told the BBC that Mr Pavitt's decision to leave was a retirement not a resignation and that it was emphatically not related to the director's decision to retire.
Still, the timing at the very least appears poor and many of the agency's critics will no doubt interpret this as a sign of crisis at the CIA, our correspondent says.