Most certainly. You should always provide some context to your replies so that people who may not have been following the thread closely, or who have other things on their minds will easily be able to determine what you're talking about.

However, when quoting, be very careful to edit the quoted sections down to the bare minimum of text needed to maintain the context for your reply. Also be careful that you clearly indicate what text you're quoting (as opposed to what you're writing), and if possible, cite the author of the original text.

If your mail program wants to attach the whole message you're replying to on the end of your replies, please do not let it do this if you can possibly avoid it. It is a good thing to include excerpts from previous messages with your replies to maintain a logical flow of discussion, but it is almost always a bad thing to include the entire text of a message being replied to, be it at the start or end of your reply.