Content
Navigating a Markdown Filesysystem
One of the things I like about Hugo is that you can set it up such that you can navigate the HEGO website in a way that mimics the underlying file structure of your markdown one of the things that we will be doing in Django, therefore, is to replicate that functionality in such a way that when we import our markdown files, the structure that they exist in on your hard drive will be mirrored in the website layout produced for your Django website. This will require you to change a number of files.
models.py
New "Folder" Model
First we want to add the following line of code to the top of homepage/blog/models.py, right under all the other import statements:
import os
import json
The next thing we will be updating is a new model for a Folder. This Python class will help us store and work with file folder information in the future. This should be added below our Tag class and above our Post Class
class Folder(models.Model):
name = models.CharField(max_length=200, unique=True,blank=True)
def __str__(self):
return str(self.name)
class Meta:
ordering = ["-date","path",]
Updates to our Post model
We also want to modify our Post class. First, we are going to rename our post's title to path, reflecting that instead of supplying a title, we are instead supplying a filepath where we will be eventually retrieving all the file's information from:
class Post(models.Model):
#title = models.CharField(max_length=200,unique=True)
path = models.CharField(max_length=200,unique=True)
You may just delete the
titleline. I commented it out here so you can see wherepathgoes
we also want to add a new folder entry, to associate posts with the folder they originated in. This data type will be of type ForeignKey since we are referencing another model in models.py, and because of the one-to-many relationship between folders and posts. So now, the top of your Post model should look like this:
class Post(models.Model):
path = models.CharField(max_length=200,unique=True)
author = models.ForeignKey(User, null=True, on_delete= models.SET_NULL)
tags = models.ManyToManyField(Tag,blank=True)
json = models.TextField(null=True)
folder = models.ForeignKey(Folder, null=True, on_delete= models.SET_NULL)
date = models.DateTimeField(blank=True,null= True)
content = models.TextField(blank=True,null=True)
We want to update the ordering based on this new data, so we need to update the Meta class:
class Meta:
ordering = ["-date", "path"]
We also want to update the Post model's __str___() method, so that it returns the post's path rather than its title (since the title has been removed for now.) So replace the old method with:
def __str__(self):
return str(self.path)
Note the only change was updating self.title with self.path.
Next, below the get_rendered_url() method, add two new methods:
def get_filename(self):
return os.path.split(self.path)[1]
def get_markdown_url(self):
path_elements = self.path.split('/')
if len(path_elements)>1:
header = path_elements[0]
dirs = self.path.split('/')[1:]
dirs[-1] = os.path.splitext(dirs[-1])[0]
dirs_string = '/'.join(dirs)
return reverse('find_post_or_folder', kwargs={'top_dir':header,'subpath':dirs_string})
if len(path_elements)==1:
header = path_elements[0]
return reverse('find_top_folder', kwargs={'top_dir':header})
def get_title(self):
try:
my_json = json.loads(self.json)
title=my_json['title']
except KeyError:
title=os.path.split(self.path)[1]
except TypeError:
title=os.path.split(self.path)[1]
return title
The first method, get_filename(), will split off the filename from the file path and return just the markdown file's name. We will use this in various spots to get the markdown file's name from its full file path when necessary.
The second method, get_markdown_url() will take a look at the post's file path and construct a url for it using Django's reverse() function. As a reminder, the reverse() function inspects a Django project's path() elements by keyword in order to construct a url that is compliant with that path's expected format and data. So therefore, get_markdown_url() first has to determine whether the post is located within a top folder of the markdown file structure or buried somewhere below, as this matches up with the two new path() elements added to urls.py below. If the post is housed within a subfolder, it will return a reverse path using the find_post_or_folder as a keyword to find the right path() generator, along with two keyword arguments: top_dir and subpath, that it compute by splitting the markdown file's path up into two parts. This reverse function will help create a url based on the full path to the post, regardless of how many folders it takes to get to the file. If the file is located within one of the top folders, however, there is no subpath to supply, so it will call a slightly different path() lookup, using find_top_folder as a search keyword, with only the top_dir kwarg supplied. In this case, the find_top_folder path() element knows to supply a blank string for subpath as we are about to see below.
The
get_markdown_url()method is part of a number of changes we need to make to enable Django to search for either posts or folders within the same url namespace. This is explained in more detail in theviews.pychanges we need to make.
Finally the get_title() method accesses the title from the imported json header information. This is required because we didn't bother to make a database entry from it
urls.py
to the bottom of our list of url patterns, add;
path('folders/', views.FolderListView.as_view(), name='folders'),
path('<str:top_dir>/<path:subpath>', views.find_post_or_folder, name='find_post_or_folder'),
path('<str:top_dir>/', views.find_post_or_folder, name='find_top_folder',kwargs={'subpath':''}),
This code adds two new ways of accessing urls to our blog. The first one combines a top directory variable name with a subdirectory path data type. This path will call the find_post_or_folder() function in views.py, with the top_dir and subpath supplied in the url.
The second path() adds support for when a user is searching for a folder at the top level. Because no subpath is supplied, we need to provide an alternate formation for that case. Note that we call the same function, but supply subpath separately as a keyword argument that is just a blank string.
The third path helps us find folders at the top of our directory. It is necessary because the subpath is not included so we need to provide that pattern as well.
views.py
We now need to add the find_post_or_folder() function called in urls.py above, as well as provide new functionality within our existing view classes for when we are accessing a Folder or Post with the particular keywords given by each path().
to the top of the file, right under all the other imports, add:
from .models import Folder
from django.http import Http404
import os
new functions
Right below this, add two new functions:
def add_parents(context, top_dir, subpath):
if subpath == "":
path = top_dir
else:
path = os.path.join(top_dir, subpath)
path_items = path.split("/")
parents = []
previous = ""
for item in path_items:
previous = "/".join([previous, item])
parents.append((item, previous))
context["parents"] = parents
context["current_folder"] = path_items[-1]
def add_children(context, topdir, subpath):
if subpath == "":
path = topdir
else:
path = os.path.join(topdir, subpath)
blocked = Folder.objects.filter(name=path)
query = (
Folder.objects.filter(name__startswith=path)
.exclude(id__in=blocked)
.order_by("name")
)
children = []
for child in query:
query2 = Post.objects.filter(folder__exact=child)
if len(query2) == 1:
if query2[0].get_filename() != "index.md":
children.append(child)
else:
children.append(child)
context["children"] = children
add_parents() will be used to identify all of a post or folder's parents, by splitting up the path by the url separator (/), and returning
add_children() will be used when displaying the contents of a folder, identifying all child folders. The exception would be folders that have only one markdown file inside named index.md. In that special case, the child folder should be treated like a post rather than a sub-folder and should not be shown in the list of children.
FolderView
Next, we need to create a new FolderView class:
class FolderView(generic.DetailView):
model = Folder
template_name = "blog/post_list.html"
ordering = [
"-date",
"path",
]
def get_object(self):
if ("top_dir" in self.kwargs) and ("subpath" in self.kwargs):
if self.kwargs["subpath"] != "":
path = "/".join([self.kwargs["top_dir"], self.kwargs["subpath"]])
results = Folder.objects.get(name=path)
else:
results = Folder.objects.get(name=self.kwargs["top_dir"])
return results
else:
raise Http404("Path does not exist")
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
if ("top_dir" in self.kwargs) and ("subpath" in self.kwargs):
topdir = self.kwargs["top_dir"]
subpath = self.kwargs["subpath"]
post_list = Post.objects.filter(folder__exact=context["folder"]).order_by(
*self.ordering
)
context["post_list"] = post_list
add_parents(context, topdir, subpath)
add_children(context, topdir, subpath)
return context
Let's look at get_object() first. This function code will do one of two things. 1) If the top_dir and subpath keyword arguments were supplied in the path, the post list will reflect only the posts in the folder at that path. Or, 2) if those arguments are not supplied, the full list of all posts on the site will be returned.
Let's now look at get_context_data(). This method adds the lists of parent and child folders generated by the add_parents() and add_children() functions defined above to the context dictionary that gets passed into the post_list.html template, so that we can permit navigation up and down the folder hierarchy from any other folder.
FolderListView
class FolderListView(generic.ListView):
model = Folder
PostDetailView
Looking at the PostDetailView class, replace it with:
class PostDetailView(generic.DetailView):
model = Post
def get_object(self, queryset=None):
if ("subpath" in self.kwargs) and ("top_dir" in self.kwargs):
fullpath = (
os.path.join(self.kwargs["top_dir"], self.kwargs["subpath"]) + ".md"
)
return Post.objects.get(path=fullpath)
elif "pk" in self.kwargs:
pk = self.kwargs["pk"]
return Post.objects.get(pk=pk)
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
topdir = self.kwargs["top_dir"]
subpath = self.kwargs["subpath"]
add_parents(context, topdir, subpath)
post = self.get_object()
return context
The updated get_object function now also performs double duty, just as the PostListView's get_queryset() method now has to. If 1) the topdir and subpath keywords are supplied, we return the post at that path. Otherwise, we return the post by its primary key (pk).
The updated get_context_data method adds the post's parent directories to its context dictionary before rendering, so that you can navigate up to a post's parents.
Finally, add our new find_post_or_folder() function to the bottom
def find_post_or_folder(request, *args, **kwargs):
subpath = kwargs["subpath"]
if subpath.endswith("/"):
subpath = subpath[:-1]
try:
return PostDetailView.as_view()(
request, top_dir=kwargs["top_dir"], subpath=subpath
)
except Post.DoesNotExist:
try:
return PostDetailView.as_view()(
request,
top_dir=kwargs["top_dir"],
subpath=os.path.join(subpath, "index"),
)
except:
try:
return FolderView.as_view()(
request, top_dir=kwargs["top_dir"], subpath=subpath
)
except:
raise Http404("Path does not exist")
This function, as mentioned up in the description for changes to urls.py, attempts to find a post at a given url. If it can't, it attempts to find a post at a given url, plus an "index" subpath, in case the url given is attempting to reach a post inside a folder. This is a special case, where we want the folder that holds that one post to act like a post. Finally, if no posts can be found, it tries to return a list of posts, provided that a folder of the same path exists. If nothing works, it raises a 404 Error.
admin.py
In homepage/blog/admin.py, add the following code:
from .models import Folder
class FolderAdmin(admin.ModelAdmin):
list_display = ['name']
admin.site.register(Folder,FolderAdmin)
This will add support for working with the folder data type in the admin panel.
Template Changes
post_list.html
Open up homepage/blog/templates/blog/post_list.html and replace the existing text with:
{% extends "base_generic.html" %}
{% block content %}
{%if parents %}
{%for item in parents%}/<a href="{{item.1}}">{{ item.0 }}</a>{%endfor%}
{%endif%}
<h1>{% if current_folder%}Posts in {{current_folder}}{%else%} Posts {%endif%}</h1>
{% if children %}
<h2>Sub-Folders</h2>
<ul>
{%for child in children%}
<li><a href="/{{child.name}}">{{child.name}}</a></li>
{%endfor%}
</ul>
{%endif%}
{% if post_list %}
{% if children %}
<h2>Posts</h2>
{%endif%}
<ul>
{% for post in post_list %}
<li>
<a href="{{ post.get_markdown_url }}">{{ post.get_title }}</a>
</li>
{% endfor %}
</ul>
{% else %}
<p>There are no posts {% if current_folder%} in {{current_folder}}. {%else%}in the blog.{%endif%}</p>
{% endif %}
{% endblock %}
We have added quite a bit of functionality. First, if the list of posts is within a certain folder, we make it possible to show the folder we are listing. Second, if there are subfolders (called children in the template), then we will add "Posts" to the listing, in order to differentiate between the list of posts in the folder and the list of subfolders. Next, we switch the url to the one generated by get_markdown_url(), which we defined above will return a url that matches the file folder structure of our markdown source. We also swap out the title for the get_filename() method we defined.
Finally, at the bottom of the post list, we also list and link to all the subfolders present in the current folder. Note the use of /{{child.name}} does not use Django's url creation functions, and will have to be fixed in future code revisions, but works for basic sites.
post_detail.html
Open up homepage/blog/templates/blog/post_detail.html and replace that template with the following:
{% extends "base_generic.html" %}
{% block title %}
{% if post.get_title %}
<title>{{post.get_title}} | DanAukes.com</title>
{% else %}
<title>DanAukes.com</title>
{% endif %}
{% endblock %}
{% block content %}
<div>
{% if parents %}<p>{%for item in parents%}/<a href="{{item.1}}">{{ item.0 }}</a>{%endfor%}</p>{% endif %}
</div>
<div>
<article>
{% if post.get_title %}
<h1>{{ post.get_title }}</h1>
{%endif%}
<hr/>
<div class="markdown">
{% with rendered_markdown=post.render_full_markdown %}
{{ rendered_markdown.html | safe}}
{%endwith%}
</div>
</article>
</div>
{% endblock %}
You can see a number of improvements from the previous version. First, we have switched from using the post's title to its path
folder_list.html
Add homepage/blog/templates/blog/folder_list.html and replace that template with the following:
{% extends "base_generic.html" %}
{% block content %}
<h1>Folders</h1>
{% if folder_list %}
<ul>
{% for item in folder_list %}
<li><a href="/{{item}}">{{item}}</a></li>
{% endfor %}
</ul>
{% else %}
<p>There are no folders in the blog.</p>
{% endif %}
{% endblock %}
base_generic.html
below the line with
<a href="{% url 'posts' %}">Posts</a>
<a href="{% url 'folders' %}">Folders</a>
<a href="{% url 'find_top_folder' top_dir='notebook' %}">Notebook</a>
This adds links on child pages to all folders and a top-level folder called "notebook".
settings.py
Add the following lines to the bottom of homepage/homepage/settings.py:
# Added to accommodate paths with slash or not
APPEND_SLASH = True