Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Jonathan Ben Avraham 43 posts 216 karma points
    May 10, 2016 @ 14:56
    Jonathan Ben Avraham
    0

    Importing Blogs from CSV file

    So I have successfully exported blogs from Umbraco, now I want to be able to re-import them from a csv file.

    The CSV file contains these columns:

    Title | Content | PublishDate | Author | Image

    This is how I'm importing the file from the backoffice:

    //This is the html view
    <div ng-controller="importBlogs">
            <div class="row">
                <div class="col-lg-2">
                    <div style="width:50%;">
                        <div id="alert2" class="alert alert-success">Import Complete</div>
                        <h2 class="importHeader" style="background: -webkit-linear-gradient(#4800ff, #ff006e); -webkit-background-clip: text; -webkit-text-fill-color: transparent; " ng-repeat="content in headers">{{content.title}}</h2>
                        <p ng-repeat="content in headers">{{content.para}}</p>
                        <input type="file" qw-single-file-upload="fileUpload" />
                        <br/>
                        <br/>
                        <button ng-repeat="content in headers" class="btn btn-primary" ng-click="importBlog()">{{content.importBtn}}</button>
                    </div>
                </div>
            </div>
        </div>
    

    This is the controller:

        //This is the js controller
        angular.module("umbraco").controller("importBlogs", function ($scope, $http) {
        $("#alert2").hide();
        $scope.headers = [{
            title: "Import All Blogs",
            para: "Click below to import all blogs",
            importBtn: "Import All Blogs"
        }];
    
        $scope.fileUpload = {};
    
        $scope.importBlog = function () {
            var uploadUrl = "/umbraco/backoffice/api/ImportAllBlogs/ImportAll";
            var fd = new FormData();
    
            fd.append("file", $scope.fileUpload);
    
    
            $http.get(uploadUrl, fd)
            .success(function (fd) {
                //HTTP OK RESPONSE
                $("#alert2").fadeIn(3000);
                $("#alert2").fadeOut(2000);
            })
            .error(function () {
                //Handle Error if Download Fail
                alert("Download Unsuccessful, try again later");
            });
        }
    
    });
    
    angular.module("umbraco").directive("qwSingleFileUpload", function () {
        return {
            restrict: "A",
            replace: false,
            scope: {
                myValue: '=qwSingleFileUpload'
            },
            link: function (scope, element, attr) {
                element.bind('change', function () {
                    scope.myValue = element[0].files[0];
    
                    if (scope.$$phase) {
                        scope.$apply();
                    }
                });
            }
        }
    });
    

    This is my C# Controller:

     using ExportUmbracoBlogsPackage.Models;
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Linq;
    using System.Net;
    using System.Web;
    using System.Web.Mvc;
    using System.Text;
    using System.Data;
    using System.Data.SqlClient;
    using Umbraco.Core.Persistence;
    using Umbraco.Web;
    using Umbraco.Web.WebApi;
    using Umbraco.Core;
    using Umbraco.Core.Models;
    using Umbraco.Core.Services;
    using umbraco.NodeFactory;
    using AutoMapper.Internal;
    
    namespace ExportUmbracoBlogsPackage.App_Code
    {
        public class ImportAllBlogsController : UmbracoAuthorizedApiController
        {
            public string basePath;
    
    
            public void LocatePath()
            {
                this.basePath = System.Web.Hosting.HostingEnvironment.MapPath(@"/upload");
            }
    
            [System.Web.Http.AcceptVerbs("GET", "POST")]
            public void ImportAll()
            {
                var myContext = Request.TryGetHttpContext();
    
                if (myContext.Success)
                {
                    HttpPostedFileBase myFile = myContext.Result.Request.Files["file"];
                    if (myFile == null)
                    {
                        throw new HttpException("invalid file");
                    }
                    else
                    {
    
                    }
                }
            }
    
    
        }
    }
    

    I'm not sure what to do next, but I'm pretty sure I should be using the ContentService or the ContentTypeService?

    Thanks in advance!

  • Steve Morgan 1349 posts 4458 karma points c-trib
    May 11, 2016 @ 08:02
    Steve Morgan
    1

    Hi,

    Yes you want the ContentService.

    A simple snippet that should get you started.. it has a simple list of objects (of type ImportProduct).. and it loops through and creates the nodes (note no duplicate checks or whatever but you'll probably want to!.

    var contentService = ApplicationContext.Current.Services.ContentService;
    // get the parent node where you want to import to
    var shopNode = Umbraco.TypedContent(1066);
    
    // simplified - I created lots of shop categories - you probably would want to do this too
    // if you do then replace shopNode.Id with with the equivalent of your category node ID (either creating it first or looking it up).
    
      foreach (var curProduct in importProducts) {
        var newProduct = contentService.CreateContent(curProduct.Name, shopNode.Id, "Product");
    
        // set each data attribute value value 
        newProduct.SetValue("ProductId", curProduct.Id);
        newProduct.SetValue("ProductName", curProduct.Name);
        newProduct.SetValue("ProductSize", curProduct.Size);
    
        // now remember to Save and publish
        contentService.SaveAndPublishWithStatus(newProductLink);
    }
    

    It goes without saying that if you are doing a large import whilst deving you should backup your database and do it in a test copy. Very annoying to create and then have to delete junk content.

    Steve

  • Jonathan Ben Avraham 43 posts 216 karma points
    May 11, 2016 @ 09:03
    Jonathan Ben Avraham
    0

    Okay so what I have thus far:

       using ExportUmbracoBlogsPackage.Models;
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Linq;
    using System.Net;
    using System.Web;
    using System.Web.Mvc;
    using System.Text;
    using System.Data;
    using System.Data.SqlClient;
    using Umbraco.Core.Persistence;
    using Umbraco.Web;
    using Umbraco.Web.WebApi;
    using Umbraco.Core;
    using Umbraco.Core.Models;
    using Umbraco.Core.Services;
    using umbraco.NodeFactory;
    using AutoMapper.Internal;
    using System.Configuration;
    
    namespace ExportUmbracoBlogsPackage.App_Code
    {
        public class ImportAllBlogsController : UmbracoAuthorizedApiController
        {
            public string basePath;
    
    
            public void LocatePath()
            {
                this.basePath = System.Web.Hosting.HostingEnvironment.MapPath(@"/upload");
            }
    
            [System.Web.Http.HttpPost]
            public void ImportAll()
            {
                var myContext = Request.TryGetHttpContext();
    
                List<BlogPostsList> postsList = new List<BlogPostsList>();
                List<BlogPosts> posts = new List<BlogPosts>();
                if (myContext.Success)
                {
                    HttpPostedFileBase myFile = myContext.Result.Request.Files["file"];
                    if (myFile == null)
                    {
                        throw new HttpException("invalid file");
                    }
                    else
                    {
                        StreamReader csvreader = new StreamReader(myFile.InputStream);
                        while (!csvreader.EndOfStream)
                        {
                            var line = csvreader.ReadLine();
                            if (line != "Title, Content, Publish Date, Author, Image")
                                line.Split();
                                List<BlogPostsList> blogPostsList = line.OfType<BlogPostsList>().ToList();
    
                                var contentService = ApplicationContext.Current.Services.ContentService;
                                var blogNode = Umbraco.TypedContent(1053);
    
                                foreach (var curBlog in blogPostsList)
                                {
                                    var newBlog = contentService.CreateContent(curBlog.Title, blogNode.Id, "Blog");
    
                                    newBlog.SetValue("Title", curBlog.Title);
                                    newBlog.SetValue("Content", curBlog.Content);
                                    newBlog.SetValue("Publish Date", curBlog.PublishDate);
                                    newBlog.SetValue("Author", curBlog.Author);
                                    newBlog.SetValue("Image", curBlog.Image);
    
                                    string newBlogLink = "/news/" + curBlog.Title + "/";
                                    newBlogLink.Replace(' ', '-');
    
                                    contentService.SaveAndPublish(newBlog, 0, true);
                                }
    
                        }
                    }
                }
            }
    
        }
    }
    

    So now the foreach loop is being hit but nothing inside it is running. Any suggestions why?

  • Barry Fogarty 493 posts 1129 karma points
    May 11, 2016 @ 09:23
    Barry Fogarty
    1

    I would use a library every time for parsing CSV files.

    Not only does it provide things out of the box to skip header rows, but it will also deal with pesky things like commas inside double quotes, etc.

    CSVHelper is my go-to CSV parsing library.

    You could then condense your code to something like (untested)

    var textReader = new StreamReader(myFile.InputStream);
    var csv = new CsvReader( textReader );
    var blogPosts = csv.GetRecords<BlogPost>();
    
    foreach (var item in blogPosts)
     { //use ContentService API to create a document for the item }
    

    Your BlogPost POCO class properties should match the headers of your CSV. Your other classes are redundant.

    As a side note, your class naming is a bit confusing, you should try to stick to singular names where possible e.g. BlogPost. A collection would then be List<BlogPost>

  • Jonathan Ben Avraham 43 posts 216 karma points
    May 11, 2016 @ 10:05
    Jonathan Ben Avraham
    0

    I've installed CsvHelper but now I'm getting an error:

    {"Fields 'Content' do not exist in the CSV file."}

    Though there is a column in my CSV file called Content...

    Why would this be happening?

  • Steve Morgan 1349 posts 4458 karma points c-trib
    May 11, 2016 @ 10:13
    Steve Morgan
    0

    Hi,

    Can you post a snippet of the first few lines of your csv.

    A few things it might be:

    1. Are you using pipes - have you set the delimiter to pipes?
    2. Are strings wrapped with ""

    I'd put my money on the delimiter - set it as per http://joshclose.github.io/CsvHelper/#reading-reading-all-records

    csv.Configuration.Delimiter = "|";
    
  • Jonathan Ben Avraham 43 posts 216 karma points
    May 11, 2016 @ 10:43
    Jonathan Ben Avraham
    0

    I figured that for some reason in my CSV each header except title was formatted like this:

    " Content", " Publish Date", " Author", " Image"

    The CsvReader was looking for:

    "Content", "Publish Date", "Author", "Image"

    whitespace is the problem here.

    My new issue is that this line is throwing the exception:

    Object reference not set to an instance of an object.

    contentService.SaveAndPublish(newBlog, 0, true);
    
  • Steve Morgan 1349 posts 4458 karma points c-trib
    May 11, 2016 @ 10:51
    Steve Morgan
    1

    You create a list

    List<BlogPosts> posts = new List<BlogPosts>();
    

    Then you read the CSV into a totally new, separate var blogPosts

    var blogPosts = csv.GetRecords<BlogPosts>().ToList();
    

    You then later iterate over the empty list posts -

     foreach (var post in posts)
    

    Just change this to

     foreach (var post in blogPosts)
    

    You can delete the posts declaration.

  • Jonathan Ben Avraham 43 posts 216 karma points
    May 11, 2016 @ 11:46
    Jonathan Ben Avraham
    0

    My new issue is that this line is throwing the exception:

    Object reference not set to an instance of an object.

    contentService.SaveAndPublish(newBlog, 0, true);
    

    my entire class looks like this:

    public class ImportAllBlogsController : UmbracoAuthorizedApiController
        {
            public string basePath;
    
    
            public void LocatePath()
            {
                this.basePath = System.Web.Hosting.HostingEnvironment.MapPath(@"/upload");
            }
    
            [System.Web.Http.HttpPost]
            public void ImportAll()
            {
                var myContext = Request.TryGetHttpContext();
                if (myContext.Success)
                {
                    HttpPostedFileBase myFile = myContext.Result.Request.Files["file"];
                    if (myFile == null)
                    {
                        throw new HttpException("invalid file");
                    }
                    else
                    {
    
                        var textReader = new StreamReader(myFile.InputStream);
                        var csv = new CsvReader(textReader);
                        var blogPosts = csv.GetRecords<BlogPosts>().ToList();
    
                         //use ContentService API to create a document for the item }
    
                        //StreamReader csvreader = new StreamReader(myFile.InputStream);
                        //while (!csvreader.EndOfStream)
                        //{
                        //    var line = csvreader.ReadLine();
                        //    if (line != "Title, Content, Publish Date, Author, Image")
                        //        line.Split();
                        //        //line[0] = postsList.Add(BlogPostsList.Title);
                        //        //line[1] = postsList.Add(BlogPostsList.Content);
                        //        //line[2] = postsList.Add(BlogPostsList.PublishDate);
                        //        //line[3] = postsList.Add(BlogPostsList.Author);
                        //        //line[4] = postsList.Add(BlogPostsList.Image);
    
                            var contentService = ApplicationContext.Current.Services.ContentService;
                            var blogNode = Umbraco.TypedContent(1053);
    
                            foreach (var post in blogPosts)
                            {
                                var newBlog = contentService.CreateContent(post.Title, 1053, "umbNewsItem", 0);
    
                                newBlog.SetValue("title", post.Title);
                                newBlog.SetValue("bodyText", post.BodyText);
                                newBlog.SetValue("publishDate", post.PublishDate);
                                newBlog.SetValue("author", post.Author);
                                newBlog.SetValue("image", post.Image);
    
                                contentService.SaveAndPublish(newBlog, 0, true);
                            }
    
                        }
                    }
                }
            }
    
  • Steve Morgan 1349 posts 4458 karma points c-trib
    May 11, 2016 @ 14:22
    Steve Morgan
    0

    Hi,

    Can you debug it? stick a breakpoint on the newBlog.SetValue("title", post.Title); line and check that Umbraco has managed to create a node.

    My guess would be that the doc type alias is wrong (umbNewsItem) or these are not allowed under 1053 or 1053 is wrong!!?? I haven't done this in a while but I also seem to remember that if you get one of the SetValue lines wrong it doesn't always throw the right error. Try just creating the titles and divide an conquer.

    I also tend to use contentService.SaveAndPublishWithStatus(); as I thought that SaveAndPublish was being deprecated (don't quote me on that though).

    Ooh as a separate thought - I've seen some weirdness when passing in an empty name - check your CSV for empty titles (and / or empty rows).

    Steve

    EDIT - what is in your image field - if that is a media picker and you're just trying to write an image url that won't work.

  • Jonathan Ben Avraham 43 posts 216 karma points
    May 12, 2016 @ 08:49
    Jonathan Ben Avraham
    0

    It was the darned image urls.

    Hmm will have to upload them manually then I guess.

  • Steve Morgan 1349 posts 4458 karma points c-trib
    May 12, 2016 @ 08:58
    Steve Morgan
    0

    You have two choices... you can create a "legacy" image location which is a simple file path string and add your images to a simple /images/ dir and then forget about them or you can use the MediaService and create them as you import the blogs.

    Personally I like to just import legacy images as static files and put a bit of logic in the template (e.g. if the user is creating a new blog post and the image picker / cropper has a value use that else fall back to outputting the file path from the simple legacy text string.

    Kevin Jump's answer here will help if you go the difficult way! https://our.umbraco.org/forum/developers/api-questions/47792-Create-Media-Programmatically-in-V7

    Good luck

    Steve

Please Sign in or register to post replies

Write your reply to:

Draft