Some legends state that Edward I intentionally built the abbey on top of Llewelyn's tomb, but that seems based more on romance than fact. The abbey was built in 1176 as a daughter house of Whitland Abbey. Despite the size of the abbey church, Cwmhir was never a wealthy monastic house. It suffered very badly in Owain Glyndwr's 1401-2 revolt.
Only the nave of the abbey church remains today. The building was originally built to a cruciform plan, but the transepts and chancel were eventually walled off. The Welsh monks at Cwmhir were once fined by Henry II for aiding a Welsh attack on English troops. At the time of the Dissolution only 3 monks lived on at Cwmhir, and there are few decent remains left of the monastic buildings.
At the east end of the church is a black slate memorial to Llewelyn set in the green turf. At the west end is a low mound which might make you scratch your head; it is not original to the castle, but was built in the Victorian period as part of the landscaping of the manor house. By the same token, the small pond to the south of the abbey church is not an abbey fishpond, as you might assume, but was part of the same landscaping of the manor house grounds.
The village of Abbeycwmhir is just about the most peaceful and secluded place you can imagine. It is set in a bowl of green hills, in an area of superb natural beauty, with the Clywedog Brook running quietly along the valley floor.
Note: If you are visiting by car, take my advice and do NOT come by way of the minor road from Rhayader. The road is exceptionally narrow and has few passing places (though it passes through very attractive countryside). Far better is the road from the north from the A483 north of Llanbister. The abbey site is open access, and is signed from the centre of the village by the church.