Right in front of the more imposing Grand palais (which hosts a nice science museum called Palais de la Découverte), the Petit Palais is a (not so) tiny and beautiful building from the 1900 Universal Exhibition. It hosts permanent collections that are not so famous but well worth seing (with remarkable paintings of Paris). In the inner courtyard is a lovely exotic garden with a cafeteria, it's the perfect place to have a break after a long and hot day walking around.
To reach the Petit Palais from Ivry you can either take line 7 till station Palais-Royal-Musée du Louvre, change for ligne 1 toward La Défense and get off at Champs-Elysées-Clemenceau, of take the RER C toward Paris, get off in Invalides and walk accross the Seine.

