Difference between Historical Romance and Historical Fiction

I met a book club yesterday that only read romances. It got me thinking. What is the difference between historical romance and historical fiction? My book has a love story in it and sex scenes, but I don’t think it is a romance. For me the difference is that the love between my characters is part of the story, but not the whole focus of the story. When I read romances the focus is almost entirely on the relationship between the two main characters and the conflicts seem larger than life. I don’t think it is a defined line though, what do you think?