Whilst at school Raff demonstrated his abilities as a polymath. Indeed, although his musical ability was evident from his earliest years, it was as a linguist and mathematician that he shone at school. Of the performing arts other than music, though, he seems to have had little aptitude or interest. His adult attitude towards the arts other than music was equivocal, judging by the description left to us by his daughter Helene in her biography of her father. For architecture, however, he had a passion as Helene Raff relates:
"Next to music it was architecture to which Raff had the strongest relationship. Occasionally an architect said to him 'If I didn't know that you are a musician I would take you for a colleague'. God knows, one would entice him and his like-minded wife with the promise of wanting to show them a beautiful architectural structure. Once their improving circumstances made it possible to take their honeymoon, [they] took the trip through southern and middle Germany called 'The Cathedral route' because on it twenty one of the most beautiful German cathedrals were to be seen.
Later, if the couple had a visit from a dear family member in prospect, one would help the other to save the necessary amount to take the guest to Oppenheim to be able to show him the famous Katharine Church there.
Raff didn't like paintings and [the] plastic arts as much as architecture; he loved it even when he wasn't extremely sure of his opinion. He occasionally handled the buildings, and sketched droll caricatures [of them] in the evenings when playing cards with his wife or sister-in-law, wittily illustrating profits or loss."
Despite being a composer with six operas to his credit, married to a well-known actress, he was no great lover of the theatre:
"Previously Raff had often spent time at the theatre in the evenings. Once he gave up his activities as a critic, he did this only seldom [but] especially if his wife appeared. Every unreasonable word expressed near to him disturbed and annoyed him, even more [so] each failure from the stage. Particularly [when] his wife made what was, in his opinion, a mistake. Finally she asked him not to go to the theatre any more when she performed and he granted her wish with great relief."