We present the highest-resolution study to date of the interstellar medium (ISM) in galaxies undergoing ram pressure stripping, using Hubble Space Telescope BVI imaging of NGC 4522 and NGC 4402, Virgo Cluster spirals that are well known to be experiencing intracluster medium (ICM) ram pressure. We find that throughout most of both galaxies, the main dust lane has a fairly well-defined edge, with a population of giant molecular cloud (GMC) sized (tens- to hundreds-of-pc scale), isolated, highly extincting dust clouds located up to ∼1.5 kpc radially beyond it. Outside of these dense clouds, the area has little or no diffuse dust extinction, indicating that the clouds have decoupled from the lower-density ISM material that has already been stripped. Several of the dust clouds have elongated morphologies that indicate active ram pressure, including two large (kpc scale) filaments in NGC 4402 that are elongated in the projected ICM wind direction. We calculate a lower limit on the H i + H2 masses of these clouds based on their dust extinctions and find that a correction factor of ∼10 gives cloud masses consistent with those measured in CO for clouds of similar diameters, probably due to the complicating factors of foreground light, cloud substructure, and resolution limitations. Assuming that the clouds’ actual masses are consistent with those of GMCs of similar diameters (∼104–105 M⊙), we estimate that only a small fraction (∼1%–10%) of the original H i + H2 remains in the parts of the disks with decoupled clouds. Based on Hα images, a similar fraction of star formation persists in these regions, 2%–3% of the estimated pre-stripping star formation rate. We find that the decoupled cloud lifetimes may be up to 150–200 Myr.