Raff was entirely, and tirelessly, self taught. Like many auto-didacts, he developed an academic streak - no doubt exacerbated by his early training in Switzerland as a schoolteacher. This expressed itself in his art with a mastery and frequent use in his compositions of counterpoint and fugue coupled with an interest in reviving the music and musical forms of the baroque and classical periods. Raff the pedagogue had to wait until his mid-30s when he moved to Wiesbaden in 1856.
There he took on private pupils and part-time music teaching posts at the two foremost girls schools in the town. This work still left him ample time to compose and his time in Wiesbaden was to prove the most productive of his composing career.
As an educator, however, Raff made his mark as director of the Music Conservatory in nearby Frankfurt. In 1877 Raff was chosen by a board advised by the composer Franz Lachner to head the city's new conservatory which was established using a bequest from a prominent musically minded citizen, Dr. Hoch. Brahms and Rheinberger had also been candidates. Leaving Wiesbaden behind, Raff was presented with an opportunity which was both daunting and enviable. Not only was he solely responsible for setting up the faculty, determining the curriculum and recruiting the staff - he even had to find a building in which to house it. In just over a year from his appointment, Frankfurt's grand new conservatory opened in September 1878.
It says something both for Raff's reputation and for the prestige of the project that he was able to entice a world class teaching staff, including two "star" performers - the pianist Clara Schumann and and singer Julius Stockhausen. Both were artistically antipathetic to Raff and very close to Brahms, who indeed helped persuade Clara to take up the post. They were joined by twelve others, including Raff's old Weimar colleague Bernhard Cossmann, the cello virtuoso and Joseph Rubinstein, the Wagner devotee. Raff taught composition.
From all accounts he made an excellent director which is perhaps surprising when one remembers his inability to cope with his own household's finances! But the institution thrived - by the end of its first year the number of students had doubled to over 120. The most famous product of Raff's years at Frankfurt was the American composer Edward Macdowell who remained devoted to Raff's memory throughout his life. His students at the conservatory seem to have been inspired by his leadership and example - he made himself as accessible as possible, saying "he who occupies an office must be like the pope in Rome, a servant of God's servant". He displayed great integrity, if perhaps not much self interest, in not using his position to promote his own music - he forbad its performance at the conservatory.
His teaching style was remembered by a student: "His instruction was expressed in a stimulating and fertile way because of his great knowledge, not in music alone but in literature, old languages, mathematics and so on. He could lecture for three or four hours, getting carried away as one subject followed another; his deep musical memory continually giving him as examples material from the works of composers from all eras. Everything flowed in its own way; admittedly he explained more by means of his sharp mind than from his heart." Another wrote "As well as canons and fugues we had to compose old and new dances without mercy. Now we had to compose a four part mens chorus, now a Sarabande or Tarantelle, now a mixed chorus. Within every period of eight days he asked for the composition of a set piece of music." Raff is portrayed as once composing a polka in C major at the blackboard "in a flash".
For his time he held a progressive attitude towards women; firstly by employing Clara Schumann and other women as teachers at the Hoch Conservatory and later by actively encouraging the admission of women students, with composition classes specifically for them - the first in Germany.
The singer Julius Stockhausen was as argumentative as Raff and had a very "artistic" temperament. Raff recognised, though, that having Stockhausen on the faculty added greatly to the conservatory's prestige and its ability to attract students. Raff paid him substantially more than his own salary as director - 9000 Reichmarks as opposed to Raff's 7000 and Schumann's 5000. As Raff said: "Of course the Director also often receives less than the primadonna. Quite simply, my primadonna is Stockhausen". Raff's primadonna repaid his generosity by involving him in a series of public disputes during 1880 which caused Raff great problems. and a period of difficult relations with the board of trustees.
Stockhausen left the Conservatory after only two years to found a competing school for vocalists.
For all the carping about his abilities as a composer Raff was universally acknowledged as a distinguished director and the position gave him great satisfaction. Nonetheless, his large workload brought on a heart attack early in 1882. He made an quick return to work but then became embroiled in a dispute with the trustees who wanted to enforce a reorganisation against his wishes. No doubt this further stress exacerbated his heart condition and his death from a second attack in June 1882 shocked staff, students and trustees alike. The whole conservatory led the funeral procession and so dear to them was his memory that, when some of the staff left the conservatory in a dispute with his successor, the rival institution which they founded was called the Raff Conservatory in his memory.