Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Erik 10 posts 51 karma points
    Jan 05, 2014 @ 21:06
    Erik
    0

    Howto: Umbraco and source control (TFS)

    Hi there,

    My company is currently looking for a relatively-cheap (€0 - €1000) CMS which we will be using for our projects.
    One of the first things that came to mind was Umbraco because of the clean backoffice interface and the responsive community.

    So i wanted to start a simple proof-of-concept website, but the first things i ran into were source control issues.
    Some questions i have:

    1. Which folders/files should be included/excluded for source control (TFS)?
      When adding everything to TFS, the Umbraco backend UI will throw lots of exceptions because TFS marks files as read-only. (when installing a package for example)

    2. When a package is installed, how does one know which files are new/added so they can be added to TFS?
      When i missed/forget some files, other team members will get exceptions on their machine.

    3. How to deal with user-generated media items?
      I thought all user-generated content was persisted in the database, but the media is stored on filesystem.
      During development team members didn't see complete content-pages.
      Images are missing and a lot of dead links to media files (.xls/.pdf/.xdoc files) because the files were fragmented over the several workstations of team members that are generating the content.

    4. Is it possible/configurable that media items will be stored in the database instead of file system?
      I think it will solve the media issues that i experienced.

      Personal opinion:
      I really don't understand why user-generated media isn't stored in the DB. (just like other website content)

      For example: a webmaster logs in the CMS, creates a news article, inserts a title, teaser, maintext, uploads image, uploads a .pdf file and links to it on the new page.

      IMHO the mentioned image and .pdf file are just user-generated content which can/should be stored in the DB. (The fact that it's an image/file/video is irrelevant).

    5. What is the simplest/easiest way for getting an Umbraco project in TFS without the mentioned issues 1,2,3 ?

      My ideal scenario would be:
      - Installing the CMS on my machine (nuget?), setup a central "development database", configure local IIS, code something, perform a (TFS) check-in excluding the /bin folder.
      -A colleague then also installs the CMS on his machine (nuget?), configures local IIS, gets latest version from TFS, runs the website.
    ---------------------------------------------------

    I left a comment related to my questions in a blogpost, and Sebastiaan Janssen was kind enough to respond.
    He advised me to look on the forum, so i searched around, scanned some posts i found, and got a bit scared.
    Lots of talk about xcopy, post build commands and manually managing package files (with regard to source control).

    First thing that came to mind was "Wow, i just want to built a simple website with my team".
    Not all my team members are .net developers (or even know about this stuff).

    I hope somemone can help me out!
    I've worked with other CMS'es in the past, but never expierenced these issues.

    Kind regards,
    Erik

  • Morten Bock 1867 posts 2140 karma points MVP 2x admin c-trib
    Jan 06, 2014 @ 12:04
    Morten Bock
    0

    Hi Erik

    In our company we also use TFS (for now), and we have a couple of different approaches to working with umbraco in source control.

    1. IIS site separate from VS
    This approach is the most stable one in my experience. What we do is to just install the UmbracoCms.Core nuget package in our VS project, which just includes the dll's. Then we keep the .zip version of the complete umbraco release in tfs on it's own. In a different folder we then set up the iis site. The rest is handled in build scripts/bat files.

    First time you want to run the site, you would then run something like "deployUmbraco.bat" which unzips the umbraco install to your IIS site folder.

    After that, you can use VS publishing to deploy your website project on top of the umbraco files.

    When doing this, you can include just the .config files you need to make changes to etc. in your tfs. Upgrading to a new umbraco version just means getting the new zip release file, updating the core nuget package to the matching version, and then run your zip extraction and publishing. (And make sure you make a compare with the config files you have in tfs to track changes in the umbraco files.

    2. Umbraco in VS folder, uncontrolled
    This is similar to the first solution, but without using IIS. In this case you set up you vs project the same way, with just the core nuget package, but you extract the umbraco files directly to your webapplication project folder. This means that you can run the umbraco site with F5 from VS.

    First time you run it, you would run the extract .zip file script, and then do a get-latest with overwrite, to make sure your config files come from TFS, and are not the default umbraco ones.

    After that, you basically just develop as usual.

    This method does have some drawbacks sine the vs project folder can be a bit of a mess, and a "rebuild" might have unintended side-effects since not all files in the folders are known by VS.

    So as I said, I prefer version 1, and with the "new" VS publishing methods and web.config transforms, it's quite easy to set up.

    Packages
    We tend to avoid most packages, because they will give problems when deploying between environments etc. But for example uComponents will mostly work with just adding the dll, which is easy to deploy, and will give you a lot of nice datatypes.

    Media
    We set up a common fileshare in dev, and point the "media" folder to that with a virtual directory in IIS. If we need to get updated content from the production site to work with, we just backup/restore the database to dev, and download the media folder to the shared folder. Then we have "real" content to work with in dev.

    You can also decide to store media in the cloud (like S3 for example). I think there is a provider model for this, but I have not tried it, and don't know how well it plays together with media pickers/croppers etc. etc.

    I hope this gives you some ideas for a setup that might work for you :)

     

  • Erik 10 posts 51 karma points
    Jan 06, 2014 @ 21:31
    Erik
    0

    Hi Morten,

    Thank you very much for your reply! A picture of the setup is starting to form in my head :)

    1. IIS site separate from VS
    I'm very interested in the solution using IIS.
    If i understood correctly, the website running in IIS is in a seperate folder outside of the solution that is binded in TFS.

    Question:
    Does that mean that if i add a new "Template" through the Umbraco UI, then i must (manually) copy this file to the TFS solution?

    Or should the "templates", "stylesheets", and "scripts" folders in the Umbraco Settings section be completely off limits for developers? (Because the changes made through the UI will not be added to the files in source control?)

    Question:
    Are there other things is should avoid editing through the Umbraco UI ?

    I'm not sure what the "deployUmbraco.bat" implementation would look like, but i'll try to google some examples.

    Packages
    I totally agree on minimizing the number of installed packages.
    Maybe i'm wrong, but i think it that installing a few packages cannot be avoided because a fresh Umbraco install lacks some basic functionality IMHO :(
    For example:
    - package for resizing images (i have never build a website that didn't need resized/cropped images)
    - package for adding icons (broad selection of icons will improve usabiliy)
    uComponents for common used datatypes
    - By default there is no way to easily tranfer database items between environments.
     Something similar to the godlike "package designer" in Sitecore. (I think/hope Umbraco's Courier is a similar tool)

    I will try to figure out if installing these packages will cause changes to existing files (or add new ones), and get them in source control somehow.

    Media
    The virtual directory options sounds pretty cool and simple. We will definitly give it a try.

    Question:
    Are there any other "gotchas" i should know about?

    Cheers,
    Erik

  • Morten Bock 1867 posts 2140 karma points MVP 2x admin c-trib
    Jan 07, 2014 @ 11:51
    Morten Bock
    100

    Hi Erik

    Yes, the iis folder is completely separate. In my case, it's even on a different machine.

    Regarding templates, what we do is to create them through umbraco, and then copy them to our VS project and include it there. To avoid having to do the manual "create template" task when deploying to stage/prod, we use the uSync package (There are different options here, depending on taste. uSiteBuilder gives you code first-ish approach)

    The deployUmbraco.bat i mentioned, just unzips the umbraco.zip file to a directory. That's all. We have a nant script for it, because the standard commandline didn't support network paths.

    Packages

    There are multiple ways to got about this. I would not rely on courier to handle deployment of packages across environments. So if the package installer needs to be run, then I'd manually install that on each environment, in order to get the DB items created. I've sometimes created a git/hg repository in the iis folder to figure out which files it touches, and then pull the needed changes/files into TFS from there. Either directly in the project, os as a folder/zip that will be deployed via script (like umbraco itself), in case of large packages.

    For image resizing, your could look at ImageProcessor, which is just a nuget package.

  • Erik 10 posts 51 karma points
    Jan 09, 2014 @ 13:32
    Erik
    0

    Hi Morten,

    Many thanks for anwersing my questions.

    I'll try your suggestions ASAP. (I hope that i can claim some R&D time tomorrow :)
    When i've done some testing i'll post my findings here and mark your reply as "the solution".

    Kind regards,

    Erik

  • Erik 10 posts 51 karma points
    Apr 03, 2014 @ 10:50
    Erik
    0

    Hi Morten,

    I still haven't got around to trying your suggestions :(

    But i'll mark your reply as "the anwer" anyway.

  • Nenad Kosanovic 18 posts 56 karma points
    Aug 19, 2014 @ 22:16
  • Marvin Varela 14 posts 35 karma points
    Jan 13, 2015 @ 19:56
    Marvin Varela
    0

    Hi all, I am new to Umbraco and the community and I am reading some of this posts in hopes to gather the general consensus concerning best practices.

    I just want to get more information regarding the specific point of packages:

    What I gather so far is that the community discourages the use of Umbraco packages and views them as "problematic" and in general the preference is to find out any other way to extend Umbraco functionality.

    Is this perception correct?

    As it has been stated on this thread Umbraco on its own seems very limited on what it can do and packages seemed, up to this point, as the natural way to extend it, but it seems they are a big "no-no"

    Please clarify as that seems a little troubling to think about, to be honest when it comes the use of the CMS.

    Thanks!

  • Dennis Aaen 4499 posts 18254 karma points admin hq c-trib
    Jan 13, 2015 @ 20:21
    Dennis Aaen
    0

    Hi Marvin,

    I will give you my thoughts about your questions and so on. Umbraco is just the shell for the content, and with Umbraco you have full control over your HTML markup, and it´s building on the .NET framework, so if you know .NET there are lots of options to extend the Umbraco CMS platform with new sections, data types and so on.

    People who think other can use things they have created can create free packages for commercial packages. An example of a package that you can use, is if you need to make a e-commerce site in Umbraco, then you can use one of the packages for e-commerce, then you don´t need to developed everthing you self. But you can of course develop this your self if you want to, but perhaps one of the exsiting packages with fit your clients needs 100% or 90 % and then you can extend this package the last 10%

    An exmple of e-commerce packages for Umbraco could be Merchello or uCommerce or Tea Commerce, as you can see Merchello is open source where uCommerce and Tea Commerce requires a license.

    But since you are new to Umbraco I would provide you some good starting ponts.The first starting point is some written documentation on the basic concepts of Umbraco, on how to create a basic website. http://our.umbraco.org/documentation/Using-Umbraco/Creating-Basic-Site/ and Blake Smith has written a newbie guide on the last years advent calendar on how to set up an Umbraco Website. http://24days.in/umbraco/2014/how-to-set-up-an-umbraco-site/

    In addition to the written documentation, you will so be able to find some videos toturials on how to work with Umbraco on Umbraco TV. The video tutorias are showing the basic concepts of working with Umbraco as well. Some of the videos are free to watch, and other requires that you subscribe to Umbraco TV. You can find the plans and prices here: http://umbraco.tv/plans-signup/

    You can find the videos about how to edit content in your Umbraco installation here http://umbraco.tv/videos/umbraco-v7/content-editor/, these videos about editing content are free to watch. You will also find free videos about how you setup the backend, with templates stylesheets, and so on here. http://umbraco.tv/videos/umbraco-v7/implementor/

    One of the milestones in Umbraco is the concept called document types. On the document types you difine which fields should be editble for the user. If you have a page document type you could for instance have fields for headline, teaser body text and so on, here are the video chapter about document types. http://umbraco.tv/videos/umbraco-v7/implementor/fundamentals/document-types/what-is-a-document-type/

    Hope this helps,

    /Dennis

  • Marvin Varela 14 posts 35 karma points
    Jan 13, 2015 @ 22:25
    Marvin Varela
    0

    Thanks Dennis, I appreciate the time you invested in writing your answer and I am grateful for you doing it so thoroughly. I also understand my question is not objective and I am basically asking for opinions rather than information, and I would understand if you feel this is not the right forum for expressing your opinion regarding my question.

    But it is clear your post does not answer my question, which was basically, whether the community would recommend the use of packages to extend Umbraco functionality or, if based on your personal experience up to this point, this is a bad practice and instead we should all just look into extend Umbraco ourselves, and what would be the reasons behind this opinion.

    Some related questions may be:

    1. Is using packages something you would only recommend to small shops building basic websites?
    2. Are packages source-control-friendly or is it better to avoid them altogether in any Umbraco site that needs to be used by a large team?
    3. Does the use of packages make website deployment complex and error-prone?
    4. Does the use of packages make it problematic for other developers on the team to build the solution after a package is added?
    5. Does the use of packages make it harder to maintain a site in the future when new developers are added to the team?
    6. Does the use of packages make it problematic to upgrade Umbraco to newer versions?
    Unfortunately, I am not coming to Umbraco in an isolated environment and I am really not free to experiment in the learning process. The use of Umbraco is a mandate in my team; of which I am just part of. We are building a Umbraco 7.2.1 for the first time. That is the main reason why I am looking for best practices from the beginning, even before I completely grasp the CMS as a whole. I really cannot just choose to use anything else, I have to use Umbraco. I just want to make sure I am not going too deep into practices that will make our lives complex and problematic in the future and this thread made me think the use of packages was one of those things, so I was asking the community for a confirmation of that.
    Your answer, although friendly and helpful seems like a way to avoid the topic instead.
  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Jan 13, 2015 @ 22:42
    Sebastiaan Janssen
    0
    1. No
    2. They are and no, don't avoid them
    3. Not really
    4. Not really
    5. No
    6. Mostly not

    Look, packages have mostly free reign over you Umbraco installation but 99.9% of the time they work great in extending the functionality of your site with little or no work on your end. In (rare!) occasions packages don't work as advertised. Look for the number of votes the packages have gotten and the compatibility reports.

    Of course, this is adding files to your site and sometimes alters your database. Files: that's what source control is for (and if you want to talk about adding complexity: TFS is not the best option). Database changes: make backups before you install packages so you can roll back. So yes, you might have to do a tiny bit of extra work (#2,3 and 6) to manage packages but it is well worth the few extra minutes of work.

    Last, but not least: packages are almost always open source so you can inspect them and see what they do. If you do have a problem with a package you can debug it and even alter it yourself if the original developer doesn't have time to update it for you.

    In my experience in the past 6 years: packages are a no-brainer. Find the ones that have good ratings and few complaints and make your life much much easier.

  • Gijs 38 posts 133 karma points
    Jan 12, 2016 @ 12:53
    Gijs
    0

    Sorry to come into this thread so late, but I just wanted to know @Sebastiaan if you have new insights to share since a year has passed since your comment.

    I mostly agree with your comment. one should not avoid packages!

    But where you answer "not really" on question 3 and 4, I would answer "Yes". And that hasn't changed since Jan 13, 2015.

    Installing packages (esp. if they come with database changes) and then deploying them to staging or production, or to build the solution on another dev-box is not where Umbraco shines.

    No hard feelings though! :D

    But if you have some new insights on how to handle this, please share!

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Jan 12, 2016 @ 13:14
    Sebastiaan Janssen
    0

    Sure thing @gijs - hence the "not really". It gets more complicated if packages add database customizations. And I would definitely contact each package owner to tell them about your experience. However, it's not just their fault of course:

    Currently the core uses something called "migrations" while upgrading your site to a newer version. Migrations take care of database updates and also allow for reverting those updates upon uninstall. We want to expose those methods publicly in a future version of Umbraco. This requires some work though as you'd need to have a similar upgrade installer UI as we currently have for upgrading Umbraco itself. There needs to be a way to run these migrations on a live site as well, using a system like App_Offline.html.

    Anyway, it IS relatively simple for package developers now to mimick this behavior, for example for this very site (Our Umbraco) we need to occasionally do some changes in the database as we're not using a UAT environment yet, nor do we use Courier yet.

    The pattern is as follows:

    • Create a eventhandler for when the site is starting
    • In the event handler look for a file called ~/App_Data/Migrations/MyMigrationName.txt and if it exists, return
    • If that file doesn't exist, these migration needs to be ran, so you can use any custom code that you want from that point onwards
    • Make sure to try/catch your code and in your finally to write the MyMigrationName.txt file so that this migration does not run again on the next app pool start
    • Do not commit the migration markers ~/App_Data/Migrations folder) to source control so that the next environment DOES actually run the migration

    An example of the source of Our can be found here (I need to implement the try/catch/finally pattern): https://github.com/umbraco/OurUmbraco/blob/cbb752c93cddb4c974321714b603638a24195626/OurUmbraco/Our/MigrationsHandler.cs

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Jan 12, 2016 @ 13:17
    Sebastiaan Janssen
    0

    Ps. Other than than, in the even farther future we want to move to having packages just be NuGet packages making deployments easier as well. But ask me again about that in another year. ;-)

  • Gijs 38 posts 133 karma points
    Jan 12, 2016 @ 13:26
    Gijs
    0

    Would be really nice if you expose those methods.

    You should definitely take a look at https://github.com/tohagan/fluentmigrator Maybe you could use that to handle migrations.

    For now the Migrationhandler is great advice. Thanks.

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Jan 12, 2016 @ 13:41
    Sebastiaan Janssen
    0

    We might have gotten some ideas from fluentmigrator, not sure where ours came from acutally. The migrations in core are pretty nice too: https://github.com/umbraco/Umbraco-CMS/blob/d50e49ad37fd5ca7bad2fd6e8fc994f3408ae70c/src/Umbraco.Core/Persistence/Migrations/Upgrades/TargetVersionSevenThreeZero/AddExternalLoginsTable.cs

  • Marvin Varela 14 posts 35 karma points
    Jan 14, 2015 @ 18:51
    Marvin Varela
    0

    Thanks a lot Sebastian, that is really helpful!

    Especially the cost-vs-benefit assessment.

    I have started with a well-known, widely-used package (imageGen) and I can see your points reflected in the installation and deployment process.

    For source control, we are using git, so things are a bit different from the original post, but most of them still apply.

    I really appreciate your answer and expert opinion!

  • Paulo Santos 11 posts 94 karma points
    Dec 12, 2016 @ 14:07
    Paulo Santos
    0

    Regarding the problems with TFS and read-only flags, I would add that we've been using TFS Local Worspaces with success, files are not set as read-only with this approach.

    Also we use the package uSync to keep the code entities (document types, data types, etc Not Content and Images) in TFS.

  • David Armitage 503 posts 2071 karma points
    Nov 19, 2018 @ 23:05
    David Armitage
    0

    Hi Guys,

    I wrote a couple of blog articles regarding source control which might help.

    1. The first is source control when using Umbraco Cloud and GitHub. https://www.umbrajobs.com/blog/posts/2018/november/umbraco-source-control-using-umbraco-cloud-and-github/

    2. The second is for people wanting to do source control without using Umbraco Cloud. For this I use GitHub with uSync to handle the database stuff. https://www.umbrajobs.com/blog/posts/2018/november/umbraco-source-control-using-github-and-usync/

    I hope this helps someone.

    Kind Regards

    David

Please Sign in or register to post replies

Write your reply to:

Draft