The Crucial Conversations authors have an excellent post about responding to a request for a recommendation that the requestee does not feel comfortable providing. Al Switzler invited input from twenty friends and summarized the findings and his recommendation. He speaks of a "fools choice" - "false dilemmas that suggest we face only two options (both of them bad), when in fact we face several choices—some of them good. We suffer from 'Or' Thinking."
Most respondents saw the situation as a forced choice between friendship and truth. Only a few respondents saw a way to honor both. One offered a suggestion that was truthful and demonstrated a deeper friendship.
I recently had a similar situation. Someone requested a recommendation that I didn't feel comfortable giving. It initiated a deeper conversation about why, and what needed to happen for me to be able to honor her request. What could have been a problem became an opportunity. We got clear and I wrote her a recommendation with my whole heart.
I like saying yes, and I like helping people. I also like being truthful. I find that every little untruth erodes my sense of self and integrity, and taking the easy way out isn't always easy. It takes more time and can get uncomfortable to honor both values. It's worth it.
Check out the article. It's good stuff.