Edit: Turns out all i needed was a night of sleep and walking through all the steps. I removed [UmbracoPicker] from the TestItem class and it worked. Thanks for taking a look at this for me though!

I'll keep everything below because it might help someone else realize they have unecessary [UmbracoPicker] attributes on their classes

This has gone through a few iterations so it looks a little different now, I got to the point where using the middle class TestModelList throws the same error as doing a direct IEnumerable

public class PageController : RenderMvcController
{
    public override ActionResult Index(RenderModel model)
    {
        var test = model.As<PageModel>();
        return base.Index(model);
    }
}

//Document type class model that is used by nested content
//doc type alias = "testItem"
[UmbracoPicker]
public class TestItem
{
    //property alias = "testProperty"
    [UmbracoPicker]
    public string TestProperty { get; set; }
}

//The "page" document type
public class PageModel
{
    //property alias = "testList"
    public IEnumerable<TestItem> TestList { get; set; }
}

The error I am getting is different now too. When I try to call test.TestList.Count() I get this error

Can't convert IPublishedContent to System.String as it has no valid constructor. A valid constructor is either an empty one, or one accepting a single IPublishedContent parameter.