Not pure compost anyway, you need at least 50% soil.
I have just been on the same job as you. An area was clear by a digger, then I went through and removed all the major rubbish (I'm talking serious stuff, roll of chain link fencing, metal drum, large lumps of trees, timber, concrete, pipes and hoses of every variety etc), then went and dug through again and took out all the scutch and bindweed. Then left it to let the weeds grow and cleared it again, taking out every bit of root. Its just about done, but its a biggish area so I have had to do it in sections, and while I was digging the second area the first area was putting up more weeds. I now have the first area covered with bark mulch (no membrane) and am selectively spraying the weeds that come through that, while avoiding the shrubs and plants. The plants to seem to do better with a bark mulch layer, however thin.
Its going to be an on-going job. Yes lift the plants and store them in a nursery area where you can keep them well watered. Dig it over again and take out every bit of root, then improve it with either new top soil or a mix of some sort of manure/chicken pellets, soil and compost. Use a combination of hoeing, selective spraying and digging up any weeds that appear. The annual weeds are easy enough, but you need to keep after scutch, bindweed and briars. Towards the autumn replant your plants. The weeds will keep coming, it will get easier but you will never get rid of them completely. As the plants and shrubs create ground cover it will cut down the weed growth considerably. And the worst of the weed season is just about over.