Kymber -
I would hope that the "auditees" appreciate and value the internal audit function that this would not be an issue. If it is an issue, the chief audit director (you?) has their work cut out and will need to change the perceptions of audit.
That said, in the past our office charged a base rate to the components of the System directly related to their expenditures plus an hourly rate for the audits conducted in the last fiscal year. Currently, that has changed to include only the hourly rate for audits.
I hope this answers your question.